Author Archive

Zookeeper RPM for Centos 7

You can download rpm for version 3.4.14 from url zookeeper-3.4.14-1.x86_64.rpm

It created from original binary from Apache Org

Used: https://github.com/id/zookeeper-el7-rpm

Centos 8 – update linux kernel

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

dnf install https://www.elrepo.org/elrepo-release-8.0-2.el8.elrepo.noarch.rpm

So, install the new kernel.

:~# dnf --enablerepo=elrepo-kernel install kernel-ml

Setup new Kernel to default

grub2-set-default 0

grub2-mkconfig -o /boot/grub2/grub.cfg

Mikrotik as OpenVPN server in 3 steps

Connect to Mikrotik via SSH and start writing text below

Set global VARs

:global COMMONNAME "openVPN"
:global COUNTRY "UA"
:global STATE "KV"
:global LOCALITY "Kyiv"
:global ORG "My org"
:global UNIT ""
:global KEYSIZE "1024"
:global USERNAME "admins"
:global PASSWORD "8charsPassword"
:global NET "77"
:global PORT "7777"

Make server and client CRT

/certificate
add name=ca-template country="$COUNTRY" state="$STATE" locality="$LOCALITY" organization="$ORGANIZATION" unit="$UNIT" common-name="$COMMONNAME" key-size="$KEYSIZE" days-valid=3650 key-usage=crl-sign,key-cert-sign
sign ca-template ca-crl-host=127.0.0.1 name="$COMMONNAME"
:delay [6]
add name=server-template country="$COUNTRY" state="$STATE" locality="$LOCALITY" organization="$ORGANIZATION" unit="$UNIT" common-name="server@$COMMONNAME" key-size="$KEYSIZE" days-valid=3650 key-usage=digital-signature,key-encipherment,tls-server
sign server-template ca="$COMMONNAME" name="server@$COMMONNAME"
:delay [6]
add name=$USERNAME country="$COUNTRY" state="$STATE" locality="$LOCALITY" organization="$ORGANIZATION" unit="$UNIT" common-name="$USERNAME" key-size="$KEYSIZE" days-valid=3650 key-usage=tls-client
sign $USERNAME  ca="$COMMONNAME" name="$USERNAME"

Create pool, profile, interface, firewall

/ip pool
add name=VPN-POOL-devops ranges="10.$NET.1.2-10.$NET.1.254"

/ppp profile
add dns-server="10.$NET.1.1" local-address="10.$NET.1.1" name=VPN-PROFILE remote-address=VPN-POOL use-encryption=yes

/interface ovpn-server server
set auth=sha1 certificate="server@$COMMONNAME" cipher=aes128,aes192,aes256 default-profile=VPN-PROFILE enabled=yes require-client-certificate=yes port="$PORT" max-mtu=1460


/ip firewall filter
add chain=input dst-port=$PORT protocol=tcp comment="Allow OpenVPN" action=accept place-before=0

add chain=input action=accept protocol=tcp src-address=10.$NET.1.0/24 in-interface-list=dynamic log=no log-prefix="" place-before=2

/ppp secret
add name=$USERNAME password=$PASSWORD profile=VPN-PROFILE service=ovpn


/certificate
add name="$USERNAME-to-issue" copy-from="$USERNAME" common-name="$USERNAME@$COMMONNAME"
sign "$USERNAME-to-issue" ca="$COMMONNAME" name="$USERNAME@$COMMONNAME"


/certificate
export-certificate "$COMMONNAME" export-passphrase=""
export-certificate "$USERNAME@$COMMONNAME" export-passphrase="$PASSWORD"

Goto menu Files in Mikrotik admin panel and download ca.crt + client.crt + client.key

Then make file SomeName.ovpn with content:

client
# в этой строчке мы указываем адрес в интернете нашего микротика
remote <your.server.IP> <Port>
dev tun
proto tcp
tun-mtu 1460

 
nobind
persist-key
persist-tun
 
keepalive 10 120
verb 3
cipher AES-256-CBC
auth SHA1

remote-cert-tls server
# эта строка задаёт файл с логином-паролем которые мы прописывали в PPP-Secrets на микротике
auth-user-pass auth.cfg

# в этой части мы задаём настройки сетей которые находятся за микротиком,
# а 172.21.108.1 это адрес микротика который мы указывали в PPP профиле

#route-method exe 
#route-delay 2 
#route 192.168.1.0 255.255.255.0 172.21.108.1

#for internet access
redirect-gateway autolocal 

<ca>
-----BEGIN CERTIFICATE-----

.....Put your ca crt from downloaded files.....

-----END CERTIFICATE-----

</ca>
<cert>
-----BEGIN CERTIFICATE-----

.....Put your client crt from downloaded files.....

-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN RSA PRIVATE KEY-----

.....Put your client key from downloaded files.....

-----END RSA PRIVATE KEY-----

</key>

Put your key and crt and change host:port vars

and file auth.cfg

user
pasword

with this two files you can start client

Configure Apache for A-rating on SSLLabs

Just add following lines to vhost.conf
SSLEngine on
SSLProtocol TLSv1.2 TLSv1.1
SSLProxyProtocol TLSv1.2 TLSv1.1
SSLHonorCipherOrder on
SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA25

Flume: java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType

error:

2017-08-09 12:29:09,754 (conf-file-poller-0) [ERROR - org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:146)] Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType
	at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:235)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.SequenceFile$CompressionType
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 12 more

Fix:

export JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre/
export JAVA_OPTS="$JAVA_OPTS -Dorg.apache.flume.log.rawdata=true -Dorg.apache.flume.log.printconfig=true "
 
HADOOP_HOME="/opt/hadoop-2.6.5"
FLUME_CLASSPATH="$HADOOP_HOME/share/hadoop/hdfs/hadoop-hdfs-2.6.5.jar"

QRencode

Забавна программа якщо треба перенести якийсь текст на телефон, а під рукою нема ніяких штатних месенджерів

echo "text_string" | qrencode -t UTF8 -o - 
█████████████████████████████
█████████████████████████████
████ ▄▄▄▄▄ █▀▄█▀▄█ ▄▄▄▄▄ ████
████ █   █ █▀▄█▀██ █   █ ████
████ █▄▄▄█ █▀▀ ▄ █ █▄▄▄█ ████
████▄▄▄▄▄▄▄█▄█ ▀ █▄▄▄▄▄▄▄████
████ ▄▄▄ ▀▄▄  ██ ▄█▄▀ ▀▄▀████
█████▀▄▄▄█▄█▀█▀▄█▀▄▀█  ▄█████
████▄▄▄▄█▄▄▄ ▀▄▄▀█ █▀▄█ ▀████
████ ▄▄▄▄▄ █▄ ▄█▄ ██▀▀ ▄█████
████ █   █ █ ▄ █▀▀ ▄▀ ▀▄▄████
████ █▄▄▄█ █ ▄▀▀▄▀▄█▀▀ ██████
████▄▄▄▄▄▄▄█▄▄█▄▄█▄█▄▄▄▄█████
█████████████████████████████
█████████████████████████████

SSH client stuck on connect

This is probably an issue with D-Bus and systemd. If the dbus service is restarted for some reason, you will also need to restart systemd-logind
restart systemd-logind service:

systemctl restart systemd-logind

I had this same issue on CentOS 7, because the messagebus was restarted (which is how the D-Bus service is called on CentOS).

Linux bridge: MAC addresses and dynamic ports

Scenario: KVM virtualization host running several bridged guests. The host has a bridge interface br0 that starts out containing only eth0, and other interfaces are dynamically added and removed from the bridge as guests are started and stopped.
The problem is, the host seems to randomly suffer some loss of connectivity (from a few to 30-40 seconds) when some guest is started or stopped. Initially one might think of something related to STP, but it turns out that it is disabled (and even then, ports appearing or disappearing should not affect existing ports).

What happens here is that, when a new guest is started, a tap interface is created and enslaved to the bridge (the tap interface is usually connected to the guest’s own ethernet interface). This interface has a name like vnet5, for example. Minor but important detail, this tap interface has a more or less random MAC address. (Note that we’re talking of the host-side tap interface, not the guest-side virtual interface, whose MAC address is configurable in the guest’s configuration file or on the command line). It is theoretically possible to set the MAC address of this host-side tap interface, but it’s quite involved and basically nobody does that (and also virt-manager has no provision to do that).

Now, by default bridge interfaces in Linux use, for their MAC address, the lowest MAC address among the enslaved interfaces. So if the newly created interface has a lower MAC, the bridge changes its MAC address and uses that of the new interface. When the guest is stopped, the tap interface is removed from the bridge and destroyed, at which point the bridge’s MAC address has to change again. Depending on when and how often guests are started/stopped, the host’s bridge interface may be changing its MAC address quite often.

Of course, this may (and does) cause connectivity problems for traffic to/from the host (fortunately, traffic to/from the guests is not affected), because neighboring computers and network devices suddenly have to update their ARP caches with the new MAC address before they can talk to the host. Depending on the traffic patterns, this may take anything from a few seconds to several tens of seconds.
Read more

Mysql: wrong slave_master_info after upgrade to 5.7

After upgrade mysql-server from 5.6 to 5.7.10 something strange going on with multimaster replication.
When you add more then one channel replication tool not see second channel and server hangs on first channel.
This happens because mysql_upgrade tool have a bug. This bug create columns in table slave_master_info in wrong order.
Read more

Apache Helix presentation

Video on youtube

Docs: http://helix.apache.org/0.7.1-docs/Tutorial.html
Zookeeper: http://zookeeper.apache.org/doc/r3.3.3/zookeeperStarted.html

Add CA certifice from Let’s Encrypt to Java 8

For add cert to Java 8 you need use keytool

cd /tmp/ && \
curl -O https://letsencrypt.org/certs/letsencryptauthorityx1.der && \
sudo $JAVA_HOME/bin/keytool -noprompt -importcert -alias letsencryptauthorityx1 -keystore $JAVA_HOME/jre/lib/security/cacerts -file  letsencryptauthorityx1.der  -storepass changeit && \
curl -O https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.der && \
sudo $JAVA_HOME/bin/keytool -noprompt -importcert -alias lets-encrypt-x3-cross-signed -keystore $JAVA_HOME/jre/lib/security/cacerts -file lets-encrypt-x3-cross-signed.der -storepass changeit

Read more