ASRock M3A UCC - FreeBSD - 2012 archive
back to main FreeBSD page for this machine.
2012 archive.
2012-12-15: slice 2 - I set up log rotation for apache log files, by adding the following lines to /etc/newsyslog.conf:
/var/log/httpd-access.log 644 9 100 * ZB /var/run/httpd.pid 30 /var/log/httpd-error.log 644 9 100 * ZB /var/run/httpd.pid 30
that's it.
2012-12-02: slice 2 - another interrupt storm:
Dec 2 03:22:52 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Dec 2 03:23:23 kg-fil last message repeated 31 times Dec 2 03:25:24 kg-fil last message repeated 121 times
Another reboot of the machine.
2012-11-25: slice2 - When I look at recorded programs in Mythweb and click on the title of the recording (or the preview picture), I get an error ("internal server error"), and Apache error log (/var/log/httpd-error.log) contains the following lines:
[Sun Nov 25 19:32:11 2012] [error] [client 10.1.150.23] Can't locate DBI.pm in @INC (@INC contains: /usr/local/lib/perl5/5.8.8/BSDPAN /usr/local/lib/perl5/site_perl/5.8.8/mach /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.8.8/mach /usr/local/lib/perl5/5.8.8 .) at /usr/local/www/mythweb/mythweb.pl line 13., referer: http://fil/mythweb/tv/recorded
When I check, it doesn't look like p5-Class-DBI is installed. Let's try to install it (I use portinstall):
root@kg-fil# portinstall -R p5-Class-DBI
when that completed (I had to locate a couple of files manually) I get another error in the Apache error log:
[Sun Nov 25 19:53:30 2012] [error] [client 10.1.150.23] install_driver(mysql) failed: Can't locate DBD/mysql.pm in @INC (@INC contains: /usr/local/lib/perl5/5.8.8/BSDPAN /usr/local/lib/perl5/site_perl/5.8.8/mach /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.8.8/mach /usr/local/lib/perl5/5.8.8 .) at (eval 7) line 3., referer: http://fil/mythweb/tv/recorded
Ok, so I need to install p5-Class-DBI-mysql:
root@kg-fil# portinstall -R p5-Class-DBI-mysql
that installed without needing any manual steps, which is good. Afterwards, the error messages in the Apache error log is gone, but now I get this in Mythweb:
1003_20121125095700.mpg does not exist in the recordings directory.
Which is strange:
root@kg-fil# l /backup/mythtv/video/1003_20121125095700.mpg* /backup/mythtv/video/1003_20121125095700.mpg /backup/mythtv/video/1003_20121125095700.mpg.png
So, is this something wrongly configured in Mythweb, or what? Hmm, the video directory is owned by root, and group users:
root@kg-fil# ll -d /backup/mythtv/video drwxrwxr-x 2 mythtv users 10752 Nov 25 11:06 /backup/mythtv/video/
and the web user (www) isn't in users group:
root@kg-fil# id -p www uid www groups www
Better fix this by adding the www user to the mythtv group, and changing group on the video directory to mythtv. Currently there isn't anyone in the mythtv group:
root@kg-fil# pw groupshow mythtv mythtv:*:119:
So add the www user:
root@kg-fil# pw groupmod mythtv -m www verify: root@kg-fil# pw groupshow mythtv mythtv:*:119:www root@kg-fil# id -p www uid www groups www mythtv
Looks good. To be on the safe side, add my own user to the mythtv group:
root@kg-fil# pw groupmod mythtv -m tingo root@kg-fil# pw groupshow mythtv mythtv:*:119:www,tingo
Next, change group on the video directory:
root@kg-fil# chown -R :mythtv /backup/mythtv
verify:
root@kg-fil# ll -d /backup/mythtv drwxrwxr-x 3 mythtv mythtv 512 Jun 1 2009 /backup/mythtv/ root@kg-fil# ll -d /backup/mythtv/video drwxrwxr-x 2 mythtv mythtv 10752 Nov 25 11:06 /backup/mythtv/video/
(and the files too). Still this doesn't fix the error. Aha - I found it:
root@kg-fil# pwd /usr/local/www/mythweb/data root@kg-fil# ls -lF total 10 -rw-r--r-- 1 root www 180 Mar 16 2007 .htaccess drwxr-xr-x 2 www www 5632 Nov 25 19:01 cache/ lrwxr-xr-x 1 www www 21 Mar 16 2007 recordings@ -> /storage/mythtv/video drwxr-xr-x 2 www www 512 Mar 16 2007 tv_icons/
Ok, let's change the recordings link:
root@kg-fil# rm recordings root@kg-fil# ln -s /backup/mythtv/video recordings
verify:
root@kg-fil# ls -lF total 10 -rw-r--r-- 1 root www 180 Mar 16 2007 .htaccess drwxr-xr-x 2 www www 5632 Nov 25 19:01 cache/ lrwxr-xr-x 1 root www 20 Nov 25 20:50 recordings@ -> /backup/mythtv/video drwxr-xr-x 2 www www 512 Mar 16 2007 tv_icons/
Does it work now? Yes (well, sort of), the link works, but my browser tries to download the file instead of streaming it. Streaming with vlc works. Good enough.
2012-11-21: slice 2 - another interrupt storm:
Nov 21 09:05:44 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Nov 21 09:06:15 kg-fil last message repeated 31 times Nov 21 09:08:16 kg-fil last message repeated 121 times
Another reboot of the machine.
2012-10-13: slice 2 - I discovered another interrupt storm again:
Oct 11 11:51:22 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Oct 11 11:51:53 kg-fil last message repeated 31 times Oct 11 11:53:54 kg-fil last message repeated 121 times Oct 11 12:03:55 kg-fil last message repeated 599 times Oct 11 12:13:55 kg-fil last message repeated 598 times Oct 11 12:23:56 kg-fil last message repeated 599 times
So I rebooted the machine.
2012-08-26: slice 2 - the theory about interrupt storms being related to recordings seems to hold: I started a recording after restarting the machine, and now I've got another interrupt storm: from /var/log/messages:
Aug 26 12:58:01 kg-fil kernel: before tuner_command! Aug 26 12:58:01 kg-fil kernel: tuner: tv freq set to 175.25 Aug 26 12:58:01 kg-fil kernel: tuner: tv 0x0d 0x63 0xce 0x02 Aug 26 12:58:02 kg-fil kernel: done waiting for cx25840 = 21 Aug 26 12:58:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Aug 26 14:37:56 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Aug 26 14:38:27 kg-fil last message repeated 30 times
The recording started at 12:58, and ended at 15:00. Why the interrupt storm started at 14:37 I do not know. Other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 11:34PM up 10:43, 3 users, load averages: 0.00, 0.00, 0.00 root@kg-fil# vmstat -i interrupt total rate irq1: atkbd0 56 0 irq9: acpi0 1 0 irq14: ata0 13139 0 irq16: pcm0 ohci0 12 0 irq17: ohci1 ohci3 2 0 irq18: ohci2 ohci4 2 0 irq20: cxm0 400713 10 irq22: xl0 atapci0 10655857073 276366 cpu0: timer 77094198 1999 Total 10733365196 278376
Now I am rebooting the machine again.
2012-08-26: slice 2 - another interrupt storm. From /var/log/messages:
Aug 25 23:13:02 kg-fil kernel: done waiting for cx25840 = 21 Aug 25 23:13:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Aug 26 01:26:26 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Aug 26 01:26:57 kg-fil last message repeated 31 times Aug 26 01:28:58 kg-fil last message repeated 121 times Aug 26 01:38:59 kg-fil last message repeated 600 times
other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 12:48PM up 6 days, 23:38, 2 users, load averages: 0.00, 0.02, 0.01 root@kg-fil# vmstat -i interrupt total rate irq1: atkbd0 54 0 irq9: acpi0 1 0 irq14: ata0 302520 0 irq16: pcm0 ohci0 12 0 irq17: ohci1 ohci3 2 0 irq18: ohci2 ohci4 2 0 irq20: cxm0 961521 1 irq22: xl0 atapci0 13535606696 22428 cpu0: timer 1206832579 1999 Total 14743703387 24430
I wonder if it is related to recordings? Anyway, I can't do anything about, so, I'm just going to reboot the machine again.
2012-08-18: slice 2 - the interrupt storm is back again (mythfilldatabase has been behaving properly since the previous entry). From /var/log/messages:
Aug 18 22:33:02 kg-fil kernel: done waiting for cx25840 = 21 Aug 18 22:33:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Aug 18 23:59:42 kg-fil kernel: interrupt storm detected on "irq20:"; throttling interrupt source Aug 19 00:00:00 kg-fil last message repeated 18 times Aug 19 00:00:01 kg-fil kernel: interrupt storm detected on "irq20:"; throttling interrupt source Aug 19 00:00:32 kg-fil last message repeated 31 times Aug 19 00:02:32 kg-fil last message repeated 120 times
other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 1:06PM up 18 days, 20:04, 2 users, load averages: 0.00, 0.00, 0.00 root@kg-fil# vmstat -i interrupt total rate irq1: atkbd0 54 0 irq9: acpi0 1 0 irq14: ata0 638738 0 irq16: pcm0 ohci0 11 0 irq17: ohci1 ohci3 2 0 irq18: ohci2 ohci4 2 0 irq20: cxm0 15714968146 9656 irq22: xl0 atapci0 24782001 15 cpu0: timer 3254396364 1999 Total 18994785319 11671
The last time it was 17 days, now 18. Funny. Rebooting the machine now.
2012-08-10: slice 2 - mythfilldatabase crashed on it's nightly run. From /var/log/messages:
Aug 11 00:58:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Aug 11 02:06:08 kg-fil kernel: pid 35124 (mythfilldatabase), uid 0: exited on signal 6 (core dumped) Aug 11 06:57:54 kg-fil su: tingo to root on /dev/ttyp1
and from gdb:
root@kg-fil# gdb mythfilldatabase /.mythtv/channels/mythfilldatabase.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `mythfilldatabase'. Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/libmythtv-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythtv-0.20.so.0 Reading symbols from /usr/local/lib/libmythavformat-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythavformat-0.20.so.0 Reading symbols from /usr/local/lib/libmythavutil-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythavutil-0.20.so.0 Reading symbols from /usr/local/lib/libmythavcodec-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythavcodec-0.20.so.0 Reading symbols from /usr/local/lib/libmythfreemheg-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythfreemheg-0.20.so.0 Reading symbols from /usr/local/lib/libmythupnp-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythupnp-0.20.so.0 Reading symbols from /usr/local/lib/libmythlivemedia-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythlivemedia-0.20.so.0 Reading symbols from /usr/local/lib/libmyth-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmyth-0.20.so.0 Reading symbols from /usr/local/lib/libmythui-0.20.so.0...done. Loaded symbols for /usr/local/lib/libmythui-0.20.so.0 Reading symbols from /usr/local/lib/libfreetype.so.9...done. Loaded symbols for /usr/local/lib/libfreetype.so.9 Reading symbols from /usr/local/lib/libmp3lame.so.0...done. Loaded symbols for /usr/local/lib/libmp3lame.so.0 Reading symbols from /usr/X11R6/lib/libXinerama.so.1...done. Loaded symbols for /usr/X11R6/lib/libXinerama.so.1 Reading symbols from /usr/X11R6/lib/libXv.so.1...done. Loaded symbols for /usr/X11R6/lib/libXv.so.1 Reading symbols from /usr/X11R6/lib/libXxf86vm.so.1...done. Loaded symbols for /usr/X11R6/lib/libXxf86vm.so.1 Reading symbols from /usr/X11R6/lib/libXrandr.so.2...done. Loaded symbols for /usr/X11R6/lib/libXrandr.so.2 Reading symbols from /usr/X11R6/lib/libqt-mt.so.3...done. Loaded symbols for /usr/X11R6/lib/libqt-mt.so.3 Reading symbols from /usr/X11R6/lib/libGLU.so.1...done. Loaded symbols for /usr/X11R6/lib/libGLU.so.1 Reading symbols from /usr/X11R6/lib/libGL.so.1...done. Loaded symbols for /usr/X11R6/lib/libGL.so.1 Reading symbols from /usr/X11R6/lib/libXmu.so.6...done. Loaded symbols for /usr/X11R6/lib/libXmu.so.6 Reading symbols from /usr/X11R6/lib/libXext.so.6...done. Loaded symbols for /usr/X11R6/lib/libXext.so.6 Reading symbols from /usr/X11R6/lib/libX11.so.6...done. Loaded symbols for /usr/X11R6/lib/libX11.so.6 Reading symbols from /usr/lib/libstdc++.so.5...done. Loaded symbols for /usr/lib/libstdc++.so.5 Reading symbols from /lib/libm.so.4...done. Loaded symbols for /lib/libm.so.4 Reading symbols from /lib/libpthread.so.2...done. Loaded symbols for /lib/libpthread.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/libz.so.3...done. Loaded symbols for /lib/libz.so.3 Reading symbols from /usr/X11R6/lib/libXrender.so.1...done. Loaded symbols for /usr/X11R6/lib/libXrender.so.1 Reading symbols from /usr/X11R6/lib/libaudio.so.2...done. Loaded symbols for /usr/X11R6/lib/libaudio.so.2 Reading symbols from /usr/X11R6/lib/libXt.so.6...done. Loaded symbols for /usr/X11R6/lib/libXt.so.6 Reading symbols from /usr/local/lib/libmng.so.1...done. Loaded symbols for /usr/local/lib/libmng.so.1 Reading symbols from /usr/local/lib/libjpeg.so.9...done. Loaded symbols for /usr/local/lib/libjpeg.so.9 Reading symbols from /usr/local/lib/libpng.so.5...done. Loaded symbols for /usr/local/lib/libpng.so.5 Reading symbols from /usr/X11R6/lib/libXi.so.6...done. Loaded symbols for /usr/X11R6/lib/libXi.so.6 Reading symbols from /usr/X11R6/lib/libXcursor.so.1...done. Loaded symbols for /usr/X11R6/lib/libXcursor.so.1 Reading symbols from /usr/X11R6/lib/libXft.so.2...done. Loaded symbols for /usr/X11R6/lib/libXft.so.2 Reading symbols from /usr/X11R6/lib/libfontconfig.so.1...done. Loaded symbols for /usr/X11R6/lib/libfontconfig.so.1 Reading symbols from /usr/X11R6/lib/libSM.so.6...done. Loaded symbols for /usr/X11R6/lib/libSM.so.6 Reading symbols from /usr/X11R6/lib/libICE.so.6...done. Loaded symbols for /usr/X11R6/lib/libICE.so.6 Reading symbols from /usr/X11R6/lib/libXau.so.0...done. Loaded symbols for /usr/X11R6/lib/libXau.so.0 Reading symbols from /usr/local/lib/liblcms.so.1...done. Loaded symbols for /usr/local/lib/liblcms.so.1 Reading symbols from /usr/X11R6/lib/libXfixes.so.3...done. Loaded symbols for /usr/X11R6/lib/libXfixes.so.3 Reading symbols from /usr/local/lib/libexpat.so.6...done. Loaded symbols for /usr/local/lib/libexpat.so.6 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000008040b09ac in pthread_testcancel () from /lib/libpthread.so.2 [New Thread 5b0800 (LWP 100092)] [New Thread 5b0400 (LWP 100149)] [New Thread 55a000 (sleeping)] (gdb) bt #0 0x00000008040b09ac in pthread_testcancel () from /lib/libpthread.so.2 #1 0x000000080409e673 in sigaction () from /lib/libpthread.so.2 #2 0x00000008040a0192 in sigaction () from /lib/libpthread.so.2 #3 0x0000000804099e46 in pthread_kill () from /lib/libpthread.so.2 #4 0x00000008040996c3 in raise () from /lib/libpthread.so.2 #5 0x000000080428d4fd in abort () from /lib/libc.so.6 #6 0x0000000803deed4e in __gnu_cxx::__verbose_terminate_handler () from /usr/lib/libstdc++.so.5 #7 0x0000000803df3106 in __cxxabiv1::__terminate () from /usr/lib/libstdc++.so.5 #8 0x0000000803df3133 in std::terminate () from /usr/lib/libstdc++.so.5 #9 0x0000000803e3114f in __cxa_pure_virtual () from /usr/lib/libstdc++.so.5 #10 0x00000008021b733e in MythContext::ClearSettingsCache (this=0x59b500, myKey= {static null = {static null = <same as static member of an already seen type>, d = 0x5618c0, static shared_null = 0x5618c0}, d = 0x591e20, static shared_null = 0x5618c0}, newVal= {static null = {static null = <same as static member of an already seen type>, d = 0x5618c0, static shared_null = 0x5618c0}, d = 0x591c40, static shared_null = 0x5618c0}) at mythcontext.cpp:1090 #11 0x00000008021b8214 in MythContext::readyRead (this=0x59b500, sock=0x4) at mythcontext.cpp:2457 #12 0x00000008022ba794 in MythSocket::readyReadThread () at mythsocket.cpp:782 #13 0x00000008040a1b49 in pthread_create () from /lib/libpthread.so.2 #14 0x00000008042007b4 in makecontext () from /lib/libc.so.6 #15 0x0000000000000000 in ?? () #16 0x00000000005b0800 in ?? () #17 0x00000008022b9560 in MythSocket::DownRef () at streambuf:499 #18 0x0000000000000000 in ?? () #19 0x0000000000000000 in ?? () #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () Cannot access memory at address 0x7fffff9fe000 (gdb)
As far as I can tell, everything else is normal:
root@kg-fil# uptime 7:01AM up 10 days, 13:58, 3 users, load averages: 0.02, 0.07, 0.03 root@kg-fil# vmstat -i interrupt total rate irq1: atkbd0 54 0 irq9: acpi0 1 0 irq14: ata0 358646 0 irq16: pcm0 ohci0 11 0 irq17: ohci1 ohci3 2 0 irq18: ohci2 ohci4 2 0 irq20: cxm0 751323 0 irq22: xl0 atapci0 1520490 1 cpu0: timer 1828328219 1999 Total 1830958748 2002
That is all.
2012-07-31: slice 2 - the interrupt storm is back again (mythfilldatabase has been failing for several days before this): from /var/log/messages:
Jul 27 04:47:02 kg-fil kernel: done waiting for cx25840 = 21 Jul 27 04:47:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Jul 31 07:19:40 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Jul 31 07:20:11 kg-fil last message repeated 31 times Jul 31 07:22:12 kg-fil last message repeated 121 times
other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 4:58PM up 17 days, 4:40, 3 users, load averages: 0.00, 0.01, 0.00 root@kg-fil# vmstat -i interrupt total rate irq1: atkbd0 56 0 irq9: acpi0 1 0 irq14: ata0 409779 0 irq16: pcm0 ohci0 11 0 irq17: ohci1 ohci3 2 0 irq18: ohci2 ohci4 2 0 irq20: cxm0 323491 0 irq22: xl0 atapci0 11529399245 7760 cpu0: timer 2970770986 1999 Total 14500903573 9761
That's it. Reboot the machine now.
2012-07-14: slice 2 - the interrupt storm is back again (the machine has been working properly in the mean time):
from /var/log/messages: Jul 13 04:57:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Jul 13 05:11:02 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Jul 13 05:11:33 kg-fil last message repeated 31 times Jul 13 05:13:33 kg-fil last message repeated 120 times Jul 13 05:23:34 kg-fil last message repeated 599 times
other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 12:16PM up 79 days, 1:10, 2 users, load averages: 0.00, 0.00, 0.00
So, just reboot it.
2012-04-26: slice 2 - the machine has been working properly since the last reboot, but now the irq storm is back again: from /var/log/messages:
Apr 23 20:27:02 kg-fil kernel: done waiting for cx25840 = 21 Apr 23 20:27:02 kg-fil kernel: mmmmmmsmmmmmmssssssss Apr 25 15:49:10 kg-fil kernel: interrupt storm detected on "irq22:"; throttling interrupt source Apr 25 15:49:41 kg-fil last message repeated 31 times Apr 25 15:51:42 kg-fil last message repeated 120 times Apr 25 16:01:43 kg-fil last message repeated 599 times
other info:
root@kg-fil# uname -a FreeBSD kg-fil.kg4.no 6.4-STABLE FreeBSD 6.4-STABLE #0: Sat Aug 13 17:30:21 CEST 2011 root@kg-fil.kg4.no:/usr/obj/usr/src/sys/FIL62 amd64 root@kg-fil# uptime 11:01AM up 104 days, 14:21, 3 users, load averages: 0.00, 0.04, 0.02
Let's see if a reboot fixes things.