Home / server / Questions / 40009
Accepted
nray
Asked: 2009-07-14 09:39:30 +0800 CST

Oracle 10g: backup failed, log disk full - how do I recover the database?

  • 772
  • Because of failed backups, the disk used for transaction logs appears to be full.
  • The database is not accepting connections, so I cannot run the (HP Data Protector) backup job manually.
  • The Oracle instance directly mounts disk from a SAN, so there isn't any file system visible to Windows.

The database service is running, however. Is there a simple way to recover the database, maybe to the point that the backup job can be re-run and restore service?

Assume no Oracle knowledge beyond tnsping ;-)

From the alert_mydatabase.log file, (<orahome>\admin\mydatabase\bdump\), 

ORA-15041: diskgroup space exhausted 
...
ARCH: Archival stopped, error occurred. Will continue retrying
...
oracle oracle10g

3 Answers

  1. Some random thoughts:

    1. Assuming you've got an Oracle Support contract, I'd get them involved. Not necessarily because this is an unusual or particularly difficult problem; but as much because of your unfamiliarity with Oracle. Get something wrong and you could well screw up your db.

    2. Why are you thinking a restore is necessary? When the disk gets full, writes to the db are "frozen" (as you're seeing), but there should be no data loss/corruption.

    3. Do you have your db sys password?

    4. Is this a live/production system?

    5. Are there any other ways of accessing the SAN filesystem?

    Repond to these questions and we can take it from there.

    • 1
  2. Best Answer

    ORA-15041: diskgroup space exhausted

    Seems like you're using ASM to store your archivelogs/backups and you have probably filled it up (probably because old backups are not getting purged)

    As cagcowboy has told you you're database is probably hung because it cannot archive and switch its current logfile but does not seem you have to recover/restore or something. Once you are able to free up some space the instance will continue its activity.

    As already suggested, I should try to get in touch with Oracle support to get proper advise.

    Regards.

    • 1

  3. Here are more random thoughts:

    1) Where is the dba?

    2) Since the database volumes are not viewable in windows, I assume the database is either using raw partitions or ASM. Do you know which one? And if not asm or raw, then what filesystem is the underlying data disk?

    3) when you backup the database, how do you handle removal/archiving of the archive logs?

    4) if not using raw/asm and using filesystem, then see if the san volume for the archive log can be extended/grow in size to allow for archive to proceed. Then immediately get do a backup and remove/archive the older archive logs.

    5) You can also try creating a new volume for archive log, get into the database and alter the db to archive to the new archive log destination.

    As far as your archive log, it appears the db has ground to halt waiting for space but has not crashed. If you make space it'll continue. Your best bet is to get a dba and a san administrator together and determine what can be done to make space. Call oracle support for help if you get stuck.

    • 1