The Backup failed

Database Backups onto disk\tape off site  remains the cornerstone of a Database Server Disaster Recovery.   Any DBA knows that having a backup strategy based on the Recovery Point Objective (RPO) is a necessary task. Not having a properly executed and regularly verified strategy is a cardinal sin. Most  aspects of database server management are excusable, but non-existent databases restores is a definite DBA error. Take the DBA Ignorance , complacency and arrogance experiment

 Daily checks on backup success\failure is one of the the first job a DBA should complete. If a failure has occurred – then investigate the root cause. Is it a disk issue? A scheduler  problem? Whatever is causing the problem – fix  as a priority

A problem in maintaining relevant backups is to manage fixing  failed backups . Out-of hrs failures magnify the problem. I’ve seen different approaches to this problem in different organisations.

 In larger companies an Operations team may exist. Typically, they will receive an alert , attempt to rerun the backup . If the backup fails , they’ll open a ticket to the DBA team.

If a DBA is on-call – they may have to rectify the issue overnight – depending on the  database priority.

The Senior DBA  should define a procedure for rectifying problems. And demand responses from Infrastructure , Storage , vendors or whoever can assist in fixing the problem.

An interesting DBA problem is the level of access to allow Operations . My preferred option is to script all aspects of backups –including instance backup or an individual database backup.  That way – I can grant limited privileges to other staff – in a documented process.

I never allow non_DBA staff to run commands – attempting to fix the problem on the database server. Such as rollforward , activate etc, it is easy to make a mistake.

A  well managed backup strategy is one of the  DBA secrets to stress reduction  

How do other DBAs manage failed backups out of hrs?