david kerr

pgfouine is a nice logfile analyzer for PostgreSQL written in php.

I’m doing a trace on a very long running ETL process and the logfile generated is ~11GB.

I’m running up against a 2GB barrier in php for fopen(). If you’ve got a 64bit machine and can recompile php with -D_FILE_OFFSET_BITS=64 then you’re good to go. But in my case, I can’t do either.

The error i’d get is:

PHP Fatal error:  File  is not readable. in /var/lib/pgsql/pgfouine-1.1/include/GenericLogReader.class.php on line 85

So for Plan B I had to remember back to the days when 64 bit wasn’t even an option (back in my day, we had 8 bits and we liked ‘em!)

I used a named pipe since pgfouine expects a file and doesn’t seem to be able to read from stdin.

mknod /tmp/pg2 p
cat /var/log/postgres > /tmp/pg2 | ./pgfouine.php -file /tmp/pg2 > bla.html

Once that kicked off I stopped getting that error and pgfouine was able to process the file.