It’s always a pain to remember the exact number of ticks needed to get the quoted output.

Here is a reminder on how to do it:

select ''''||schemaname||'.'||tablename||'''' from pg_tables

4 quotes when there’s no text, or 3 quotes on the outside and one on the inside when there is text.

select '''public.'||tablename||'''' from pg_tables

I recently ran into a problem like that in PostgreSQL, the query ran fine *most* of the time, but every once in a while it would go off into the weeds.

After a little digging I realized that the table’s statistics hadn’t been updated in quite a while. This struck me as odd since I knew that PostgreSQL automatically updates statistics on a regular basis. So I did some digging to figure out exactly what triggers statistics to be updated in PostgreSQL.

First off, it’s important to note that in postgresql the vacuum process is what handles statistics. “vacuum analyze” is the command although you can just use “analyze” too. Vacuuming and analyzing are different operations, and have different, but very similar parameters that control them.

The key postgresql.conf parameters that affect whether or not a table gets auto-analyzed are:

#autovacuum_analyze_threshold = 50
#autovacuum_analyze_scale_factor = 0.1

(You can see the defaults in the postgresql.conf above)

The formula that Postgres uses to determine if a table needs to be auto-analyzed is
([ # of rows in table ] * [ scale factor ]) + [ threshold ]

So, for example, if you have a table with 10,000 rows in it, then it would look like this:
( 10000 * 0.1 ) + 50 = 1050

So if you were to add 1049 rows, the table would not get auto-analyzed.

So how do you change it? Well, there are 2 ways. you could change the 2 parameters in the postgresql.conf, and this is a good idea if you think you need to globally lower or raise them.
However, I think in most cases you’ll want to make the change just for one table, which means you need to utilize the pg_autovacuum table.

postgres=# \d pg_autovacuum
    Table "pg_catalog.pg_autovacuum"
      Column      |  Type   | Modifiers
------------------+---------+-----------
 vacrelid         | oid     | not null
 enabled          | boolean | not null
 vac_base_thresh  | integer | not null
 vac_scale_factor | real    | not null
 anl_base_thresh  | integer | not null
 anl_scale_factor | real    | not null
 vac_cost_delay   | integer | not null
 vac_cost_limit   | integer | not null
 freeze_min_age   | integer | not null
 freeze_max_age   | integer | not null
Indexes:
    "pg_autovacuum_vacrelid_index" UNIQUE, btree (vacrelid)

Remember when I mentioned above that vacuuming and analyzing were handled via similar mechanisms? you can see that here, the table has fields related to vacuuming and analyzing.

The fields we care about for this particular problem are: vacrelid, enabled, anl_base_thresh and anl_scale_factor.

• vacrelid is the OID of your table. you can get that via:

select oid from pg_class where relname = [tablename]

• enabled is “true”
• anl_base_thresh is your new autovacuum_analyze_threshold
• anl_scale_factor is your new autovacuum_scale_factor

To make the parameter change for just one table we then need to insert into the pg_autovacuum table.

postgres@postgres=# select oid from pg_class where relname = 'test_vac';
   oid
---------
 1579952

postgres@postgres=# insert into pg_autovacuum(1579952,true,50,0.2,50,0.09,20,-1,100000000,200000000)

I’m changing the scale factor from 0.1 to 0.9, the rest are the default values for those parameters
What that means is in our example:
( 10000 * 0.1 ) + 50 = 1050
Changes to:
( 10000 * 0.09 ) + 50 = 950

In other words, the autovacuum would kick off 100 records sooner, not a huge difference, I know however to the planner it could make all of the difference in the world.

The reason I wanted to use HA-JDBC is that I was looking for a no-coding required solution for seamless fail over. (Similar to Oracle RAC) for PostgreSQL.

I’ll be using a Shared Disk / Heartbeat cluster on the server side, however when the node fails the application will register an error, which is undesirable.

After doing more research it’s been determined that HA-JDBC won’t work.

It seems that HA-JDBC is, at best, a SQL replicator, where you have 2 active nodes and HA-JDBC will perform inserts and updates across both databases to keep them in sync. This is fine if you’re not using the “serial” data type in PostgreSQL, Triggers, Functions, time based default values, etc. Using any sort of trigger, the code fires independently on insert and you end up getting out of sync databases.

Another reason why HA-JDBC won’t work is that if a node is un-reachable HA-JDBC removes it from consideration. So your 2nd “standby” when it becomes active won’t be considered by HA-JDBC without some manual intervention.

And finally, I’d advise steering away from HA-JDBC at this point even if the above works for you. I posted a few clarifying questions regarding the above to both the HA-JDBC forums and their mailing list and received no response. If your business is looking into true high availability for their servers you need to choose all of your components with care. A non-responsive community either means the project is dead, or un-caring, both of which are unacceptable when you’re looking into HA solutions.

Finally got a response from Paul Ferraro, Nice guy, very helpful posted below:

Sorry for the slow response...

HA-JDBC is not the right tool for this job.  HA-JDBC is an
*alternative* to shared disk failover - it was not designed
to be used in concert with it.  Instead, you want something
like JBoss HA DataSource or Weblogic multi-pools.  These are
DataSource proxies whose getConnection() method returns a
raw connection from the first available data source.
DataSource-level.  Connections returned by HA-JDBC's DataSource,
on the other hand, are proxies to connections to each active
database in your cluster.

I can go into more detail if you'd like, such as the advantages/
disadvantages of HA-JDBC over shared-disk failover, if you're
interested.

Paul

I’m doing a trace on a very long running ETL process and the logfile generated is ~11GB.

I’m running up against a 2GB barrier in php for fopen(). If you’ve got a 64bit machine and can recompile php with -D_FILE_OFFSET_BITS=64 then you’re good to go. But in my case, I can’t do either.

The error i’d get is:

PHP Fatal error:  File  is not readable. in /var/lib/pgsql/pgfouine-1.1/include/GenericLogReader.class.php on line 85

So for Plan B I had to remember back to the days when 64 bit wasn’t even an option (back in my day, we had 8 bits and we liked ‘em!)

I used a named pipe since pgfouine expects a file and doesn’t seem to be able to read from stdin.

mknod /tmp/pg2 p
cat /var/log/postgres > /tmp/pg2 | ./pgfouine.php -file /tmp/pg2 > bla.html

Once that kicked off I stopped getting that error and pgfouine was able to process the file.

Without updating the JDBC driver you would have to catch the failure at the java container level or in the app itself and manage the switch from the down node to the active node.
I don’t think that’s “industry standard” and it’s certainly not easy by any means.
The normal way is to let the JDBC driver manage it.

Unfortunately the PostgreSQL JDBC driver doesn’t handle this event out of the box so we need to invoke a 3rd party.

There aren’t a lot of options in this area here are two:

• HA-JDBC
• Tungsten SQL Router which used to be called Sequoia

I found a good discussion around HA-JDBC here

I’m using Hibernate + Geronimo so i need to do testing to see if that’s going to work with HA-JDBC but from the sounds of it, it should work just fine.

I’ll need to evaluate both of these to determine which is the best.

The reason I’m looking at clustering is that the DB server will be handling a large number of users and any downtime is catastrophic. So reliability comes before any performance or administrative concerns in a clustering solution.

My platform is PostgreSQL 8.3 and SLES Linux.

I looked at 4 Solutions:
Option 1: Shared Disk (Heartbeat) Cluster (Heartbeat: SLES)
Option 2: Filesystem Replication Based (DR:BD / GNDB)
Option 3: DB Replication Based (Slony I)
Option 4: DB Replication Based (PGCluster)

I weighed the pro’s and con’s of each of them and eventually chose Option 1 as the best for my needs.

I like the heartbeat solution because:

• It’s simple
• There’s no data loss in a shared disk cluster
• There’s no replication overhead so no performance impact

Unfortunately, there is very little public documentation regarding heartbeat clusters used with PostgreSQL. I hope to rectify that over the next weeks and months, so stay tuned.

Set your agent home

AGENT_HOME=/opt/app/oracle/agent10g

Make sure $AGENT_HOME/bin is in your PATH
Run the agentca with the reconfigure option. (you may need to specify a –c for RACs)

vexbert<11> agentca -h
Usage: agentca -d/-f [-nctis]
        -d  option for rediscovery.
        -f  option for reconfigure.
        -n  to specify the CLUSTER_NAME.
        -c  specify the list of cluster nodes separated by commas
        -t  don't start the agent after rediscover or reconfigure.
        -i  oraInst.loc location on Unix/Linux Platforms only.
        -s secure the Agent during RAC/Cluster Agent reconfiguration
         (Don't use this option in upgraded home).

vexbert<11> agentca –f

Note: you may need to clean up duplicates in your OEM after running this on an existing system.

#!/usr/bin/perl -w

use IPC::SysV qw(IPC_PRIVATE IPC_RMID S_IRWXU);

#$size = 314580992;
$size = 50000000;
$id = shmget(IPC_PRIVATE, $size, S_IRWXU) || die "$!";
sleep 10;
shmctl($id, IPC_RMID, 0)

kerrd@vexbert<10> ipcs -bm
IPC status from  as of Thu Oct 30 10:42:06 PDT 2008
T         ID      KEY        MODE        OWNER    GROUP      SEGSZ
Shared Memory:
m 1493172237   0x32d14fb8 --rw-r-----   oracle      dba 2097160192
m 1979711547   0x5e000b64 --rw-------     root     root        512
m  721420366   0xe867eba0 --rw-r-----   oracle      dba 2147491840
m         25   0x67e97344 --rw-r-----   oracle      dba  419438592
m         21   0xc4986724 --rw-rw----   oracle      dba  629153792
m         19   0x71e930c0 --rw-rw----   oracle      dba 1073758208
m          7   0x4154520e --rw-rw-rw-     root   tivoli    1048576
m          6   0x4154520a --rw-rw-rw-   tivoli   tivoli    1048576
m          5   0x4154520d --rw-rw-rw-   tivoli   tivoli    1048576
m          4   0x4154520c --rw-rw-rw-   tivoli   tivoli    1048576
m          3   0x4154520f --rw-rw-rw-   tivoli   tivoli    1048576
m          2   0x4154520b --rw-rw-rw-   tivoli   tivoli     524288
m          1   0x41545200 --rw-rw-rw-   tivoli   tivoli    1048576
m          0   0          --rw-------     root     root         64

The Oracle/DBA segments are related to Oracle database instances.
The SEGSZ is limited by the Unix kernel.

sudo /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.

There is a bug in 10g where, on reboot when CRS auto-starts a database, the database will be in a “hung” state.

The only way to solve this is to:

• Kill the pmon process (to hard crash the database on all nodes).
• Restart the databases manually (one at a time)

To avoid this, remove the DBs from the oratab before reboots on a cluster/RAC DB.