Repairing CODB Database Corruption

In rare cases, the CODB database used by CCEd can become corrupted. This is uncommon, but when it happens, it usually appears in one of two ways:

OID Mismatch

The file /usr/sausalito/codb/codb.oids contains a list of Object IDs currently marked as being in use.

If this file no longer matches the actual objects that exist, problems can occur. In many cases, this issue can be fixed by carefully editing this text file.

db.classes Corruption

The file /usr/sausalito/codb/db.classes is a Berkeley DB v1.85 database created when CCEd is installed. CCEd uses it to track classes, objects, and their indexes.

If this database becomes corrupted, older recovery options were very limited. In the past, the usual conclusion was that the system could only be salvaged through migration, and even that was not always successful.

The RPM package blueonyx-db now includes tools to diagnose and repair both types of CODB issues.

#> rpm -ql blueonyx-db
/home/solarspeed/db/bin/db_load185
/usr/sausalito/sbin/examine_codb.pl
/usr/sausalito/sbin/repair_codb_classes.pl

Usage

/usr/sausalito/sbin/examine_codb.pl

This tool generates a report and shows whether the CODB index is consistent or damaged.

/usr/sausalito/sbin/repair_codb_classes.pl

When run with the --fix parameter, this tool stops the following services:

crond
cced
admserv
cced-api

It then creates a timestamped backup of the CODB directory, for example:

/usr/sausalito/codb-backup-20260518-153802.tar.gz

The repair process uses the Berkeley DB v1.85 tool db_load185, which is included in the blueonyx-db RPM:

/home/solarspeed/db/bin/db_load185

This is the same Berkeley DB version that CCEd uses internally, which is important for compatibility.

The tool dumps db.classes, adjusts the contents to match the actual objects present in /usr/sausalito/codb/objects/, and then rebuilds the database.

After the repair is complete, the previously stopped services are restarted. At that point, CODB should be usable again.

Example: Broken Server

#> /usr/sausalito/sbin/examine_codb.pl
MISSING_OBJDIR  DnsRecord       1222    /usr/sausalito/codb/objects/1222
NOT_IN_CODB_OIDS        DnsRecord       1222
MISSING_OBJDIR  DnsRecord       1224    /usr/sausalito/codb/objects/1224
NOT_IN_CODB_OIDS        DnsRecord       1224
MISSING_OBJDIR  DnsRecord       1162    /usr/sausalito/codb/objects/1162
NOT_IN_CODB_OIDS        DnsRecord       1162
MISSING_OBJDIR  DnsRecord       1223    /usr/sausalito/codb/objects/1223
NOT_IN_CODB_OIDS        DnsRecord       1223
MISSING_OBJDIR  DnsRecord       1221    /usr/sausalito/codb/objects/1221
NOT_IN_CODB_OIDS        DnsRecord       1221
MISSING_OBJDIR  DnsSOA  1219    /usr/sausalito/codb/objects/1219
NOT_IN_CODB_OIDS        DnsSOA  1219

SUMMARY
dump_file=/tmp/db.classes.dump
entries=1758
duplicates=0
missing_object_dirs=6
class_mismatches=0
oids_not_marked_in_use=6

This server has six objects listed in /usr/sausalito/codb/db.classes, but the corresponding object directories no longer exist under /usr/sausalito/codb/objects/. In this example, all affected objects are DNS records.

Repairing the Database

#> /usr/sausalito/sbin/repair_codb_classes.pl --fix
live_objects=1752
live_class_entries=1752
class_oids_mismatched=0
current_codb_oids=1752
clean_dump=/tmp/db.classes.clean.dump
codb_backup=/usr/sausalito/codb-backup-20260518-153802.tar.gz
RESULT codb_backup=/usr/sausalito/codb-backup-20260518-153802.tar.gz db_backup=/usr/sausalito/codb/db.classes.bak.20260518-153802 replaced=1 db_file=/usr/sausalito/codb/db.classes oids_file=/usr/sausalito/codb/codb.oids

Verifying the Repair

#> /usr/sausalito/sbin/examine_codb.pl

SUMMARY
dump_file=/tmp/db.classes.dump
entries=1752
duplicates=0
missing_object_dirs=0
class_mismatches=0
oids_not_marked_in_use=0

The summary now shows no missing object directories, no class mismatches, and no Object IDs missing from codb.oids. This indicates that the CODB database is consistent again.