Setting up master-slave replication for mysql

 

So, I had to set-up master-slave replication. Here’s how you do it:

Why would one want to set-up this? Easy.

1)Spread the read load across multiple servers:  you can use the MySQL master for writing (inserts, updates) and the slave(s) for reading (SELECT) This should speed up things nicely for your application.

2)Do the backups from the slave. You might have noticed that backing-up, especially big databases, slows things down. That happens because mysqldump locks tables as it reads them. This can slow a big site or even takes it down from a few seconds to a few minute. Reading from the slave affects nothing. You can even stop the slave, read the /var/lib/mysql folder and start it back.

Following are the steps on how to set it up, but first let’s not a few assumptions. Also please note that this sets up only the basic things. Many more settings can be tweaked to improve MySQL performance.

Master server ip: 192.168.0.1
Slave server ip: 192.168.0.2
Slave username: slaveusr
Slave pw: slavepass
Your data directory is: /var/lib/mysql

Add the following lines in your master my.cnf file under [mysqld] section:

[cce]
# master my.cnf
server-id = 1
relay-log = /var/lib/mysql/mysql-relay-bin
relay-log-index = /var/lib/mysql/mysql-relay-bin.index
log-error = /var/lib/mysql/mysql.err
master-info-file = /var/lib/mysql/mysql-master.info
relay-log-info-file = /var/lib/mysql/mysql-relay-log.info
datadir = /var/lib/mysql
log-bin = /var/lib/mysql/mysql-bin
# end master 
[/cce]

The following settings are for the slave’s my.cnf under the same [mysqld] section:
[cce]
# slave’s my.cnf
server-id = 2
relay-log = /var/lib/mysql/mysql-relay-bin
relay-log-index = /var/lib/mysql/mysql-relay-bin.index
log-error = /var/lib/mysql/mysql.err
master-info-file = /var/lib/mysql/mysql-master.info
relay-log-info-file = /var/lib/mysql/mysql-relay-log.info
datadir = /var/lib/mysql
# end slave settings
[/cce]
Restart both mysql instances after changing the settings in my.cnf

Create the slave user on master server:
[cceN]
mysql> grant replication slave on *.* to slaveusr@’10.0.0.2′ identified by ‘slavepass’;
[/cceN]
Dump the full data and copy it (scp, etc.) to the slave

[cceN]
mysqldump -u root -pROOTPASS –all-databases –single-transaction –master-data=1 > masterdump.sql
[/cceN]

import this dump on the slave server:

[cceN]
mysql -u root -pROOTPASS < masterdump.sql
[/cceN]

After dump is imported go in to mysql client by typing mysql. Let us tell the slave which master to connect to and what login/password to use:
[cceN]
mysql> CHANGE MASTER TO MASTER_HOST=’192.168.0.1′, MASTER_USER=’slaveusr’, MASTER_PASSWORD=’slavepass’;
[/cceN]

Start the slave:
[cceN]
mysql> start slave;
[/cceN]
 

Check the status on the slave:
[cceN]
mysql> show slave statusG
[/cceN]
The last row will tell how many seconds the slave runs behind the master. No worries if it’s not 0, it should catch up quickly (at that time it will show Seconds_Behind_Master: 0) If it reads NULL, maybe the slave is not started (you can start by typing: start slave) or it could be that there is a problem(shows up in Last_errno: and Last_error under show slave status).

Make sensors report temperature on slackware64-current with an Intel I7 2600k

For some reasons the required modules don’t get loaded. Add the following to /etc/rc.d/rc.modules right above the line reading ### Mouse support:

[CCE_BASH]
### by SV for i2600k temperature
/sbin/modprobe coretemp
/sbin/modprobe pkgtemp
[/CCE_BASH]

Now sensors report the temperatures.

Recursively remove digikam’s xmp embeded tags

Because digikam made a mess of the tags I started to add to my pictures I decided to start from scratch. Note that I tried to get the max and I had set to write tags both to .xmp sidecar files and inside the image files. I also use the mysql backend.

One of the above broken everything (I started seeing _digikam_root_tag) and some of my tags were doubled. So all in all I wanted to delete them all:

  • Remove your collection from digikam
  • Drop all tables from the digikam database
  • Remove all .xmp files. In the folder containing your collection:
    find . -name *.xmp -exec rm {} ;
  • Remove all embedded xmp tags (again in the folder containing you collection)
    exiftool -r -P -xmp:TagsList= -xmp:LastKeywordXMP= -xmp:HierarchicalSubject= -xmp:Subject= -iptc:Keywords= *

DON’T FOLLOW THIS BLINDLY

This is just a sketch of the process. If you don’t understand what are you doing don’t do it! I also kept all .original files exiftool generated, just in case I discover something later on.

 

Unpacking a .deb file (including scripts)

While attempting to install a deb package on a slackware system I wanted to check the install script that the package runs. I tried the deb2tgz utility but unfortunately that only left me the files in the package in a ready to install .tgz file.

Here’s how to unpack a .deb to obtain both scripts and files:

[cce_bash]
cd /tmp/somefolder
ar x somepackage.deb
[/cce_bash]
This leaves you with two files:

control.tar.gz and data.tar.gz The first contains the pre and post install scripts and the second contains the files.

Log dropped packets using CentOS Firewall

Please take care modifying your firewall. If you don’t understand what’s being done here you may lock yourself out of your machine. You’ve been warned 🙂 !

In order to log dropped packets on the INPUT chain I replaced this:
[cceN_bash]
-A INPUT -i eth0 -j REJECT –reject-with icmp-host-prohibited
[/cceN_bash]
with this
[cce_bash]
-N LOGDROP
-A LOGDROP -i eth0 -j LOG
-A LOGDROP -i eth0 -j REJECT –reject-with icmp-host-prohibited
-A INPUT -i eth0 -j LOGDROP
[/cce_bash]
Of course you can use DROP instead of REJECT –reject-with icmp-host-prohibited

Fixing a self pressing button on an Asus VW202SR monitor

Well, today instead of working I had the pleasure of fixing the monitor 🙂 The monitor is an older Asus VW202 SR that my wife uses connected to the laptop. I tried everything else (power cycling, another signal cable, another computer) but it turns out to be really the button. The issue manifested itself by the brightness setting window appearing and going eventually all the way to 100. First I thought that the button was stuck in the frame but it was safely depressed. That means a broken button on the pcb board. I thought it would be pretty easy to solder a new one in place.

Read more “Fixing a self pressing button on an Asus VW202SR monitor”

Enabling the MySQL slow query log

If you have a slow web app or website it might be because of unoptimized sql queries. You want to track and fix those and the mysql slow query log is very helpful here.

Enabling the slow query log

MySQL prior to 5.1.0 requires adding a setting to the MySQL my.cnf file and restart the mysql daemon in order to log slow queries; from MySQL 5.1.0 onwards you can also change this dynamically without having to restart.

To make the setting permanent every time you start MySQL (or you have a version prior to 5.1.0) add (ow uncomment) the following in my.cnf (usually /etc/my.cnf or /etc/mysql/my.cnf)
[cceN_bash]
log_slow_queries = /var/log/mysql/mysql-slow.log
[/cceN_bash]
You can skip the path or leave it blank completely and mysql will create the log in the default location. The default is to log the queries into a file in the MySQL data directory.

To enable or disable the setting dynamically in MySQL 5.1.0 run the following query to enable it:
[cceN_bash]
set log_slow_queries = ON;
[/cceN_bash]
and to disable it:
[cceN_bash]
set log_slow_queries = OFF;
[/cceN_bash]

Changing the long query time

You can also set how long a query needs to take before it’s considered a “long” query. The default is 10 seconds. I usually like to set it lower:

To change it to 5 seconds add in my.cnf:
[cceN_bash]
long_query_time = 5
[/cceN_bash]
This can be changed dynamically in MySQL 5.0.0+ (and possibly earlier versions) by running the following query:
[cceN_bash]
set global long_query_time = 5;
[/cceN_bash]
This will only work for new connections; any connections which have already been established will continue to use the old setting. Once the user disconnects and reconnects their new connection will use the new setting.

Errors you can get applying these settings dynamically

If the following error message appears when attempting to change the log_slow_queries setting dynamically means you are using a version of MySQL that does not support changing the setting dynamically:
[cceN_mysql]
ERROR 1193 (HY000): Unknown system variable ‘log_slow_queries’
[/cceN_mysql]
The long_query_time value must be integer; if it’s not (e.g. you set long_query_time = 2.5;) then you’ll see this error:
[cceN_mysql]
#1232 – Incorrect argument type to variable ‘long_query_time’
[/cceN_mysql]
Note also that if you set the long_query_time to 0 it will not fail, but the actual setting applied will be 1 and not 0.

Limit number of related articles returned in Joomla 1.7 and Joomla 2.5

Joomla 1.7 has a nice module mod_related_items (Articles Related) that shows articles related to the current one the visitor is browsing. It has one issue though, you can’t set a limit.

So a little code hacking is in order. Open modules/mod_related_items/helper.php and go to line 94 (right after the Filter by language if) and modify $db>setQuery($query); to read:
[cceN_php]
$db->setQuery($query,0,10);
[/cceN_php]
Of course you can change the limit from 10 with whatever you like. Random order is commented out because using ORDER BY RAND has performance issues. If you think you can handle it feel free to uncomment.

This hack also works for Joomla 2.5!

Homework:

  • add parameter in the admin interface Feel free to post the solution 🙂

cbnet Ping Optimizer is killing a site

Some wise soul decided many moons ago that installing the aforementioned plugin on a site I host is a great ideea. The wise soul is long gone (admining websites for ngo’s as a volunteer is no fun for far too many come and goers:) ) Lately the site was getting more and more sluggish with no apparent reason. I decided to clean a bit the many modules he thought in his wisdom to install. Like any major work on a site I started with a backup. I was puzzled to see that mysqldump would hang. I checked, it was not hanging, it was dumping a really huge .sql file. That was not normal as the site only has a few hundreds of articles. A check in phpmyadmin later and I found the culprit:

WPPREFIX_cbnetpo_ping_optimizer table had over 11Gb. Great, 82 million records for some errors about not being able to ping. Great: no limit on how much you log in the table, no throttling on errors (each error is logged tens of times in the same second) and so on. I could track where this comes from but I simply removed the darn plugin and dropped the table.

So great idea on installing untested plugins and great coding by the plugin author.

Cadsoft Eagle on Slackware Current 64bit

I want to start building some new electronics projects and I wanted to use some tool to build my schematics. Even if not open source Cadsoft’s Eagle seems to be the de-facto standard between hobbyists.

Unfortunately trying to run the setup on Slackware gives the errors regarding to
[ccNe_bash]
/…/bin/eagle: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory
[/ccNe_bash]
libssl.so.1.0.0 and libcrypto.so.1.0.0

I run a multilib setup so that’s not it. It turns that all I had to do was:
[cce_bash]
cd /lib
ln -s libssl.so.0.9.8 libssl.so.1.0.0
ln -s libcrypt-2.14.1.so libcrypto.so.1.0.0
[/cce_bash]
in /lib

Hope that helps.