MySQL server has gone away

I recently began getting failed MythTV recordings a few times per week (MythTV version 0.23.1/MySQL version 5.1.47). When I looked in the mythbackend log I found that there was some underlying database connectivity issue. The logs would fill up with error messages like these:

2011-01-16 05:26:30.699 Error preparing query: SELECT chanid, useonairguide FROM channel, dtv_multiplex WHERE serviceid=:SERVICEID AND networkid=:NETWORKID AND transportid=:TRANSPORTID AND channel.mplexid=dtv_multiplex.mplexid AND channel.sourceid=:SOURCEID
2011-01-16 05:26:30.699 Driver error was [2/2006]:
QMYSQL3: Unable to prepare statement
Database error was:
MySQL server has gone away
2011-01-16 05:26:30.700 DB Error (Looking up chanID):
Query was:
SELECT chanid, useonairguide FROM channel, dtv_multiplex WHERE serviceid=:SERVICEID AND networkid=:NETWORKID AND transportid=:TRANSPORTID AND channel.mplexid=dtv_multiplex.mplexid AND channel.sourceid=:SOURCEID
Bindings were:
:NETWORKID=42249, :SERVICEID=3, :SOURCEID=1, :TRANSPORTID=5
Driver error was [2/2006]:
QMYSQL: Unable to execute query
Database error was:
MySQL server has gone away

The system did eventually recover (within an hour or so), but all recordings during this time would fail to start. Interestingly enough, they would still manage to get inserted into the table of recordings. To me this indicates that there is still partially functioning database connectivity. Even though the message says that MySQL server has gone away, I could perfectly well connect to the database during the time that mythbackend was filling my logs with these kind of errors. My conclusion was that the problem was with the database connections between mythbackend and MySQL, not with MySQL itself.

So, I began investigating what could be the cause of these errors. What I found was that MySQL has a timeout on all database connections that’s by default set to 28800 seconds (8 hours).

To see your connection timeouts, do this:

mysql> SELECT @@global.wait_timeout, @@session.wait_timeout;
+-----------------------+------------------------+
| @@global.wait_timeout | @@session.wait_timeout |
+-----------------------+------------------------+
|                 28800 |                  28800 |
+-----------------------+------------------------+
1 row in set (0.00 sec)

Mythbackend deadlocks and auto-restart script

I recently bought two new DVB-C cards (two Technotrend C-1500). Until then I was using Anysee E30 Combo Plus. So now I have three cards altogether. As always, Linux and DVB hardware is a whole story in itself, I’ll write more about this upgrade in a later post. But anyway, I eventually got all three cards working together.

However, there seems to be some deadlock issue in mythbackend when you use multiple DVB cards. I’ve previously used multiple tuners for one card and never had an issue with that configuration. After I upgraded to three cards, every now and then (weekly) when a recording is ongoing and a new one is about to start or has just started, MythTV will deadlock. What I mean by deadlock is that it becomes completely unresponsive from mythfrontend and mythweb. Logs stop. There’s no particular error in the logs that would indicate any kind of problem. I do get an entry for the recording in the database, but the file it is pointing to doesn’t exist.

I found a ticket that sounds suspiciously similar, but the issue was fixed in revision 22814. According to the release notes, that fix should be in MythTV 0.23, which I’m using. Also, since the recording entry is inserted into the database, one might assume that it is not a scheduling issue?

I’m hoping this issue will go away with future upgrades, like these issues usually do. To handle the situation for now, I made a script to automatically restart mythbackend if it deadlocks or dies. This is a good idea to have even if your setup is relatively stable. You don’t want mythbackend to die shortly after you leave for that two-week vacation, and thereby miss all recordings, do you?

This is the mythbackend deadlock restart script I made:

#!/bin/sh
# print status
STATUS=`/sbin/service mythbackend status | grep -E "mythbackend .*is running"`
echo `date`: $STATUS
FILEDATE=`date +%Y%m%d_%H%M%S`
mv -f /tmp/mythbackend-status /tmp/mythbackend-status.previous
RESULT=`wget --timeout=10 --output-document=/tmp/mythbackend-status --retry-connrefused --tries=4 http://127.0.0.1:6544 2>&1`
if [ "$?" -ne "0" ]
then
        mv /tmp/mythbackend-status.previous /tmp/mythbackend-status.crashed.$FILEDATE
        echo -- ERROR: mythbackend is not responding... must restart: $RESULT
        echo `/sbin/service mythbackend status`
        echo `/sbin/service mythbackend stop`
        echo `/sbin/service mythbackend start`
        echo `date`: Restart done
else
        echo Service is responding: $RESULT | sed "s/Saving.*//"
        rm /tmp/mythbackend-status.previous
fi

MythTV EIT data not working

After I did a new channel scan in MythTV 0.23.1 I discovered that I didn’t get any EIT data from the DVB-C network anymore. I’m in Helsinki (Finland) and using Welho cable provider. So, I basically went through the same process as this guy, creating channels.conf files and so on. Eventually I discovered his page, which solved the issue of not getting EIT data. I did some more research into this and how to potentially solve it in other ways.

Solution #1 – Do a Full Scan (Tuned)

So, what worked for me was what he describes in his post, i.e. to do a Full Scan (Tuned) with the following parameters:

Frequency: 162000000
Modulation: QAM_64

When I look closely, those are the values for a few of the channels in my channels.conf file:

4Sport:162000000:INVERSION_AUTO:6900000:FEC_NONE:QAM_64:417:673:145
The Voice/TV Viisi:162000000:INVERSION_AUTO:6900000:FEC_NONE:QAM_64:513:660:241
DINA:162000000:INVERSION_AUTO:6900000:FEC_NONE:QAM_64:641:642:64000
4SportPro:162000000:INVERSION_AUTO:6900000:FEC_NONE:QAM_64:418:675:193
Infokanava.:162000000:INVERSION_AUTO:6900000:FEC_NONE:QAM_64:221:222:1973

MythTV 0.24 mythtvosd is broken

I did an upgrade from MythTV 0.23.1 to 0.24 a few weeks after 0.24 had been released. I eventually ended up reverting the whole upgrade. There were a few reasons:

• One of my recordings failed. This happened only once, a few hours after the upgrade. My recordings practically never fail otherwise. I would estimate that about 1 out of 500-1000 recordings fail under normal circumstances. This got me a bit suspicious and I decided it was better to revert as quickly as possible. It could of course have been a coincidence, but I didn’t want to take the risk. Once you’ve had the system up and running for a few days, it’s really to late to (easily) revert the upgrade unless you are prepared to lose all the changes in the database (recordings etc).
• Another issue in 0.24 was that the mythtvosd utility doesn’t work, see bug report. For those of you not familiar with the utility, it allows you to send text popups to mythfrontend, which are then shown on the screen. I use it to show warnings about low disk space and various other remote controlled system information popups (I might write a post about this later). Apparently they are not planning on fixing it until 0.25 :-( This may mean that I will have to skip 0.24 altogether. If so, it would be the first time I skip a whole MythTV release. Let’s see…

Anyway, I didn’t really see any big must-haves in version 0.24. So, for the moment I’m probably better off with a stable 0.23.1. Of course, it might be that the current 0.24 version is more stable than the one I tried.