Page 13 of 14

PG_REORG Utility for VACUUM FULL online

pg_reorg is a utility made by NTT for reorganizing the table structure.

Concept is simple, if you have all the require pointers and data in same page, then accessing those is much more faster. This is what pg_reorg provides to a user.

Following are some options, which pg_reorg provides.

-o [ —order-by] columns

This option makes pg_reorg to oraganise the table data as per the mentioned column. At the backend pg_reorg will creates a new table using CTAS and SELECT Query include ORDER BY clause with columns mentioned with -o.

-n [—no-order] tablename
[/cod]
When this option is being used, then pg_reorg, does the VACUUM FULL ONLINE. Now, question is how it must be doing. Simple Concept, create a new table using CTAS and create a trigger on current table to track the DML. As the New table got created play those tracked DML on new table. It works well. This option is only for table which has primary key.

pg_reorg by default does the CLUSTER of tables and it follows same concept, i.e without locking table do the CLUSTER.

After performing all the options, pg_reorg does the ANALYZE on the table.

Following are some elog information, which it performs at backend:

elog(DEBUG2, "---- reorg_one_table ----");  
elog(DEBUG2, "target_name    : %s", table->target_name);  
elog(DEBUG2, "target_oid     : %u", table->target_oid);  
elog(DEBUG2, "target_toast   : %u", table->target_toast);  
elog(DEBUG2, "target_tidx    : %u", table->target_tidx);  
elog(DEBUG2, "pkid           : %u", table->pkid);  
elog(DEBUG2, "ckid           : %u", table->ckid);  
elog(DEBUG2, "create_pktype  : %s", table->create_pktype);  
elog(DEBUG2, "create_log     : %s", table->create_log);  
elog(DEBUG2, "create_trigger : %s", table->create_trigger);  
elog(DEBUG2, "create_table   : %s", table->create_table);  
elog(DEBUG2, "delete_log     : %s", table->delete_log);  
elog(DEBUG2, "lock_table     : %s", table->lock_table);  
elog(DEBUG2, "sql_peek       : %s", table->sql_peek);  
elog(DEBUG2, "sql_insert     : %s", table->sql_insert);  
elog(DEBUG2, "sql_delete     : %s", table->sql_delete);  
elog(DEBUG2, "sql_update     : %s", table->sql_update);  
elog(DEBUG2, "sql_pop        : %s", table->sql_pop);  
Interesting Hunh.

Some Comparison between clusterdb and pg_reorg is given in below link:
http://reorg.projects.postgresql.org/index.html

With everything, DBA has to take care of few things:
While pg_reorg is going on one table, the DBA should not let anyone to execute DDL Changes and Index on the currently reorganizing table.

Slony Vs PG9.0 Built in Streaming Replication.

>People Generally asked such kind of Questions as PG9.0 Comes with Streaming Replication.

Following are some points which people need to think before deciding which replication, they should follow:

1. Slony has a some overhead on database than the Streaming replication+HotStandby in 9.0
2. All the changes must be apply via SLONIK Command
3. Slony gives advantage of replicating some tables and allows to ignore others
4. Slony also gives the advantage of replication between Different version of PG and PG on different OS.

PG9.0:: Monitoring Hot Standby

Now PG9.0 is in Market with new feature of Hot Standby and Streaming Replication.

So, I have started to explore the way of monitoring the Hot Standby. I was in process of writing my own code for Monitoring the Hot Standby.

For this purpose I have written a shell script to find the way of calculating lag.

In pgpool-II, Developer has used following formula to calculate the lagging:

lsn = xlogid * 16 * 1024 * 1024 * 255 + xrecoff;  

Following is an explanation of meaning of xlogid and xrecoff:

postgres=# select pg_current_xlog_location();  
  
pg_current_xlog_location   
--------------------------  
0/13000078  
(1 row)  
0: is xlogid and xrecoff is 13000078

With this, Concept of implementation of finding the lagging is to calculate the replication lag by comparing the current WAL write location on the primary with the last WAL location received/replayed by the standby. These can be find using pg_current_xlog_location function on the primary and the pg_last_xlog_receive_location/pg_last_xlog_replay_location functions on the standby, respectively.

Following is also a code made by Karl Denninger:
http://www.denninger.net/check-replication.c

Above program, check the status of Master and Slave, if Both are in sync then it will be silent, if there is lagging then it will give proper message to stdout.

pgFouine: PostgreSQL Log Analyzer

PgFouine is a interesting PostgreSQL Analyzer Tool which is available for Generating the Reports in html format.

Advantage of using this tool is that user gets report in text or HTML format, which is easy to analyze.

Using pgFouine user can make following types of Reports:
1. Error Reports
2. Slow Query reports
3. Vacuum Verbose Reports etc…

Installation of pgFouine is simple.

Download the source from following location:
http://pgfoundry.org/frs/download.php/2575/pgfouine-1.2.tar.gz

and then extract the pgfouine source using following command:

<pre class="cpp">tar zxvf pgfouine-1.2.tar.gz.</pre>

Please note, before using pgfouine user has to make sure that it has php installed on his server.

pgFouine has some restriction over analyzing the log file. It analyzes the PostgreSQL logfiles, if the log_line_prefix has following format:

<pre class="cpp">log_line_prefix = 'user=%u,db=%d ' ( Filter on database with user with syslog )
log_line_prefix = '%t [%p]: [%l-1] ' ( For standard errors)
log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d ' (Filter on database/user with steer)</pre>

For more information on Installation and configuration of pgfouine, following is a link:
http://pgfouine.projects.postgresql.org/tutorial.html

Usage of pgfouine:

<pre class="cpp">php pgfouine.php -file  -logtype [syslog|stderr|csvlog] > report.html</pre>
<pre class="cpp">

pgfouine can also be used for analyzing the vacuum verbose output.

Following are sample of VACUUM VERBOSE report generated by pgfouine.


Physical Standby Vs Hot Standby

Some thoughts always come in mind about standby terminologies. Once, Someone has asked question about Physical Standby of Oracle10g.
Is Oracle10g Physical Standby a Hot Standby Or WarmStandby?
Till Oracle 10g, Physical Standby of Oracle is a standby which has two mode: 1. Managed Recovery Mode 2. Read Only Mode. It cannot be in both mode at same time.
If the standby is in recovery mode then, it’s a Warm Standby and when its read only mode then, then it’s lagging from Primary and would be able to response SELECT queries.
Either way, till Oracle 10g physical standby was not meeting the requirement of Hot Standby. It was playing the role of Warm Standby.
However, from Oracle 11g, Physical Standby can be Recovery mode and read only mode, both at the same time, which now fulfill the definition of Hot Standby. With the complexity of managing the Standby, licensing thing comes in Picture.
PG9.0 onwards, PostgreSQL now has Cold Standby, Warm Standby and Hot Standby, with zero cost. Community is making the PostgreSQL Technology more advance with new features.

Hot Standby in PostgreSQL 9.0:

As per definition of Hot Standby, its a method of redundancy in which primay and secondary (Backup Server) runs simultaneously. The data is mirrored to secondary so that both should contain identical data in real time. With this user, would be able to execute the Query against Database while secondary is in archive recovery mode.

This is what introduce in PG9.0. In-built Hot Standby.

PostgreSQL Community always try to make the things simpler as much as possible, same they have proven in Hot Standby Implementation.

Following are the steps of configuring Hot Standby:

1. Make sure following parameters are set in Configuration file, postgresql.conf of Primary:
wal_level = ‘hot_standby’archive_mode = onarchive_command = ‘cp %p /Library/PostgreSQL/9.0/data/archivelog/%f’

2. After setting the above parameters, Now take the hot backup of PG9.0 Instance. Steps of Hot Backup is simple:
a) Execute following command in Primary:

select pg_start_backup(‘Hot Standby Backup’);

b) Take the file system backup of PG 9.0 Data directory, as given below:


cp -r $PRIMARY/* $HTStandby/

c). Execute following command to stop the Hot Backup:

select pg_stop_backup();

3. After taking the Hot Backup and restoring it desired location, set following parameter in postgresql.conf file of Hot Standby:

hot_standby = on

4. Create a recovery.conf file in Hot Standby restore location and set the following parameters:

standby_mode = 'on'
restore_command = 'cp /Library/PostgreSQL/9.0/data/archivelog/%f %p'

5. Clean the pg_xlog and pg_xlog/archive_status directories of Hot Standby Data directory.
6. Remove the postmaster.pid file from Hot Standby Data Directory.
7. Now start the Instance using following command:

pg_ctl -D $HTStandby start

Now, hot standby is up and running.

Following are few snapshots (Primary Port is 5432 and Hot Standby Port 5433:

1. Here I have inserted one value in table “test” and created a new table test2 with some records:


2. Now on Standby Following is a snapshot:

pgAgent Configuration on Windows

Here are the steps which some one can use to configure the pgAgent. These steps are tested on my machine:

1. Login to system as test_user:

Please create a pgpass.conf file in test_user %APPDATA%\postgresql directory:

Entry for pgpass.conf should be as given below:

hostname:port:database:username:password

2. Connect to database as given below:

C:\”Program Files”\PostgresPlus\8.4\bin\psql.exe -U postgres

(Above command will also verify that pgpass.conf is used by the psql command or not)
3. After connecting to database, create plpgsql language as given below:

CREATE LANGUAGE plpgsql;

4. Now, run the pgagent.sql file.

5. Create pgAgent service, as given below:

pgagent.exe INSTALL pgAgent -u test_user -p test hostaddr=<host> dbname=<dbname> user=<username>

3. Start the created service using following command:

net start pgAgent

4. While Creating job, Please keep the output of following command in “Host Agent” field of “pgAgent Job” window:

select jagstation from pgagent.pga_jobagent;