Back to the Basement Computing Page
 

The Basement Blog
 

Introduction
 
Recent Entries
Acrobat Printing Problem
Vista - Missing Sidebar
Update Problem - XP SP3
Vista Problem-XPS M1330
Matlab Mex Problem
Vista and Telnet
Vista and ASF
FAT32 and Windows XP
Vista Annoyances - GUI
Vista - Installing
IE7 Java Crash
Win 98 Update Problem
Speeding up Windows
Outlook Reminders
Flash Update Problem
Intermittent Boot
Invisible Hard Drive
USB Drive Quirks
Outlook .pst Backup
Windows Defender Crash
XP Print Dialog Crash
Mac OS 10.4 printing
 
Archives
2008
2007
 
Contact Us
 
Feedback
 
Disclaimer
 
Backing up Outlook .pst Files
Tuesday, April 3, 2007

(The following has only been tested on Windows XP SP2)

Microsoft Outlook (not to be confused with Outlook Express) is a very common and useful program that is widely used for email and calendaring. In fact it is the program I use myself, and have come to rely on it.

It is very important that Outlook data (and email in general) be backed up on a regular basis - I would suggest at least once a day.

Outlook stores its data (email, calendar etc.) in two main types of files: .ost and .pst files. These files are by default in the folder "C:\Documents and Settings\<username>\Local Settings\Application Data\Microsoft\Outlook", where <username> is the user name you normally use to log-on to your Windows PC. However, the user may change this location via an option in Outlook, so you should check where these files are for a particular instance before backing them up (see refs. # 1 and 2 below).

"ost" stands for Offline STorage, and this file is normally a mirror of data stored on the Exchange Mail Server. "pst" is short for Personal STorage, and the .pst files contain data that is on the local PC alone (normally). It is very important that the .pst file(s) be backed up on a regular basis. I am assuming the .ost data is mirrored on the server and is being backed up by the server administrator.

The problem

Until recently, I was backing up my files to another disk using XCOPY (see ref. # 3 below). I placed a command similar to the following in a batch (.bat) file and scheduled it to run once every day, after midnight:

 xcopy c:\*.* z:\ /c/f/h/d/y/o/s/e /EXCLUDE:exclude.txt 
      
The "Exclude.txt" contains a list of files and folders I did not want backed up (such as "Temporary Internet Files").

This simple backup works very well - it quickly copies files that have changed since the last backup to the destination disk (Z drive in our example).

I noticed a potential problem early on with this method - certain files would not get copied with an "access denied" message. These are files currently open on the source (C:) drive, e.g. certain system files, or a document currently being edited. Initially, I ignored the problem, thinking that it was not necessary to backup system files anyway, and the same user file was very unlikely to be "open" for more than a day or two at a stretch, and most backups ran at night, when most people would not have any open data/document files.

There is a problem with this line of thought - some people tend to leave Outlook running at night, so the Outlook .pst files are in danger of not getting backed up.

And there is an even worse problem. When XCOPY tries to copy a file, it first checks if a destination file of the same name exists and deletes it, it then tries to copy the source file to the destination. This is very bad for backups, because it means that if you have Outlook open, then XCOPY will delete the previous backed up copy, and then not be able to copy the new version because it is "open". Thus you lose the old backup if you happen to have Outlook open at any time during a future backup! This problem applies to all files backed up with XCOPY, not just Outlook files, but the consequences are greatest for Outlook data because of the way Outlook stores everything in one or two big .pst files, and because Outlook tends to be left running more often than other applications. At best, I would call this a very bad design on the part of whoever wrote XCOPY.

I started looking for alternatives to XCOPY. I restricted myself to free options because this is something I would like to use on other Windows PC's as well and did not want to pay for it.

Some Alternatives

The first option I considered was ROBOCOPY (ref. # 4). ROBOCOPY (available as part of the Win2003 Resource Kit Tools - downloadable from http://www.microsoft.com/downloads/details.aspx?FamilyID=....) is a command line program similar to XCOPY, but reportedly more "robust" and with additional options.

Unfortunately, testing showed that ROBOCOPY suffers from the same flaw, i.e. if it tries to copy a file that is currently open, it first deletes the destination file, and is then unable to copy the "open" source file. This results in a loss of the older backup. This is no improvement over XCOPY for copying Outlook .pst files.

Next, I looked at XXCOPY (ref. # 5). XXCOPY is another command line program with many extensions and improvements over XCOPY. It comes as a free version for use on a personal computer, or as a paid site license for more extensive use. I tried the free version.

A quick test showed that XXCOPY does not have the same problem as XCOPY or ROBOCOPY. It correctly does not delete the destination file if the source file is open.

However, when I tried to run XXCOPY to backup to a remote network disk, a new problem appeared. When starting, it now displayed a message like "You are attempting to copy over the network with the free version of xxcopy. You may want to get the site license. Press return to continue.." (or words to that effect). If you press the Return or Enter key, the program does resume, but this makes it useless for unattended backups. There seemed to be no way to disable this warning, and I wanted a free solution, so I gave up on XXCOPY.

A web search took me to a page titled "How to automatically back up your personal folders file in Outlook 2000 and Outlook 2003" (ref. # 6). This describes an Outlook add-in from Microsoft named pfbackup, and has a link to a free download. It seemed very promising, so I gave it a try.

The test showed that this method is not very automatic. What it does is to offer to backup the .pst files each time the user exits Outlook. This makes it not so suitable for those users who rarely exit Outlook, or the network drive may not be available when the user does exit, or the careless (i.e. average) user may decide to ignore the reminder, etc. It still may be useful if you want a more interactive backup, and I have included a link below for how to use it (ref. # 7).

The solution

For the final solution, I turned to that old standy - NTBackup (ref. # 8). NTBackup is a program that is included with all recent versions of Windows. It has a GUI that can be accessed via Start -> All Programs -> Accessories -> System Tools -> Backup. It can also be accessed from the command line. I had been avoiding using it because it likes to store backed-up files in a special archive format, with all files going into a single archive file (named e.g. backup.bkf). This takes away some of the simplicity of XCOPY, which saves each file separately with its original folder and file name.

Windows XP and 2003 or later include a feature called "Volume Shadow Copy", which allows certain programs to make copies of open or in-use files, among other things(ref. # 9). It turns out that NTBackup is one of the programs that can take advantage of Volume Shadow Copy, and that makes it very useful for this purpose.

I decided to use NTBackup for backing up the Outlook data folder, and XCOPY for everything else.

Using NTBackup from the command line is a bit murky, and it turns out one of the files (the .bks file) needed by the command line must be created by the GUI version of NTBackup, so I started by launching NTBackup (Start -> All Programs -> Accessories -> System Tools -> Backup). If it starts in Wizard mode, un-check the box that reads "Always start in Wizard mode", and click on "Advanced Mode".

Fig. 3

Click on the "Backup" tab as shown above. This brings up a screen similar to the following:

Fig. 4 - click to enlarge

In the left window, expand the tree to find the folder that contains the Outlook .pst files. By default, they would be in "C:\Documents and Settings\<username>\Local Settings\Application Data\Microsoft\Outlook", but in our case they are in c:\ravi\data\email, so I checked that box, and made sure all others were un-checked. Next I entered the folder and file name to be used for backups - Q:\Ravi\Outlook\Outlook.bkf. You will probably want to use another location or name, but it must end in .bkf. (Drive Q: is the network drive where the backup will reside, we should have mapped this already, and later we will map each time from the backup batch file.)

Don't change anything in the right-side window, and click on "Start Backup". This brings up the window shown below:

Fig. 5

We plan to over-write the old backup each night, so I selected "Replace the data....". I also entered some suitable comments that describe the backup. Then click on "Schedule". It will prompt to save the backup selection, click "Yes" and then select a file name Outlook.bks for saving in the default folder named "Data", then enter your password when prompted. The next window asks for "Job Name", enter "Temp" (without quotes) for the job name, then click "Properties". The next window has a "Run On" field that you should modify to select a date in the future, e.g. tomorrow. (Don't worry about the times or other properties at this time, we will modify later). Click OK, OK, Cancel etc. and exit the NTBackup program.

By the above, we used the NTBackup GUI to create a scheduled task. The backup task has not run even once yet, and we don't plan to let it run. Instead, we will use it to extract certain information as follows:

Click Start -> Control Panel -> Scheduled Tasks. This brings up the list of tasks, which should include the job named "Temp" that we created above.

Fig. 6

Notice that it has never run. Right-click on the job icon for Temp and select "Properties". This brings up the Properties window, shown below:

Fig. 7

Highlight everything in the "Run" field, type CTL-C on the keyboard to save it, then open a new blank window with Notepad, and press CTL-V to paste the contents of the "Run" field there. At this point the Notepad window will contain something like the following:

Fig. 8

Leave the Notepad window open, but close/cancel the Temp Properties window, then right-click on the Temp job in the Scheduled Tasks list and select "Delete" and click "Yes" when it asks to confirm.

Next, make the following changes in the Notepad window:

  • Add a command to map the network drive (Q:).
  • Change the comments in the backup command itself to be more meaningful.
  • Add a command at the end to disconnect the network drive (Q:)
  • Add any comments
Of course the commands to map and disconnect the network drive are only necessary if the backup is to be saved on a network drive. In my case, the contents of the Notepad window now look like this:

Fig. 9

You may notice that I included the xcopy command to backup all other files as well. This way we just need to schedule one job to do both things.

Next, save the Notepad file in a relevant folder with a name like Daily-Backup.bat. I saved in a folder named c:\backups

The next thing is to schedule this batch file so it runs nightly.

Start -> Control Panel -> Scheduled Tasks, and double-click on "Add scheduled task". Click "Next" on the first dialog. There is a pause of from 10 to 60 secs, and then the following window appears:

Fig. 10

Click on "Browse" and then locate the Daily-Backup.bat file we just saved above (in our case in c:\backups) and then click "Open". The next window appears:

Fig. 11

Click on "Daily", then click "Next".

Fig. 12

Select time and starting date. Here we chose to run the backup at 12:30 AM every night. Click "Next". In the next window, enter your password twice. You can also choose a username different than the one you're logged in as.

Click "Next" and then "Finish" and make sure the job named "Daily-Backups" appears in the Scheduled Tasks list after a few seconds.

At this point we are almost done.

Test, Test, Test

The world's best backup is worthless if you discover that you cannot find or use it when there is a real emergency. It is therefore very important indeed to check your backups carefully initially, and then periodically thereafter.

(to be continued...)

References

  1. Where Outlook Stores Your Mail... and other data http://email.about.com/cs/...
  2. Managing Outlook Folders in Office 2003 http://www.quepublishing.com/articles/...
  3. XCOPY reference http://www.microsoft.com/resources/.../xcopy.mspx
  4. ROBOCOPY - Robust Copy http://en.wikipedia.org/wiki/Robocopy
  5. XXCOPY - File Management Utility http://www.xxcopy.com/
  6. How to automatically back up your personal folders file in Outlook 2000 and Outlook 2003 http://support.microsoft.com/kb/238782/
  7. Using the Microsoft Outlook Personal Folders Backup tool
    http://office.microsoft.com/en-us/outlook/HA010875321033.aspx
  8. NTBackup http://www.microsoft.com/.../ntbackup_command.mspx
  9. Windows XP and 2003 volume shadow copy service
    http://searchwincomputing.techtarget.com/tip/..._gci1076710,00.html

Comments
(in order from older to newest)

"Thank You very much for the article!"

April 16, 2007, 2:33 AM by Andres L.

"This article just solved us a problem with backing up outlook files."

Aug. 9, 2007, 11:56 AM by Mario V.

"It took too much time to backup the pst file. I decided to take a look at incremental PST backup from http://www.datamills.com"

Aug. 16, 2007, 11:18 AM by Wayne S.

"As regards backup i don't know, but for restore- how to open ost file in outlook, our personal information will be lost, because Microsoft Exchange Server cannot read from old *.ost files, when an account is recreated. In this case, *.ost file should be recreated too, have a possibility to open ost file in Outlook 2003 and recover your data from these encrypted *.ost files, program is very easy to use, when you open ost file in new account,there are two recovery options, first of all, you can move your *.ost file to a *.pst file, the last one can be easily opened by Microsoft Outlook or any other compatible email client."

Feb. 26, 2008, 6:19 AM by Alex K.

"Just a great workaround to my problem."

Nov. 13, 2008, 8:57 AM by Thierry

"I wrote a macro in Excel to do the automatic backup with copy and NTBACKUP. It works fine with XP 2003, but I get an access denied with XP 2002. It worked for a while with XP 2002, but after a windows update, I guess, it started systematically to give an access denied message. I have to do it by hand."

Sep. 16, 2009, 11:07 AM by Antoni G.

"I hear that EdgeSafe will soon backup regular files as well, so it will provide a full workstation backup... on top of the incremental PST Backup (http://www.datamills.com)"

Mar. 5, 2010, 10:13 AM by Mario V.

Leave a comment
(We reserve the right to edit or delete any comment considered offensive or off-topic)

Subject:

 
Your Name: (or just initials - this will appear on the web page)

 
Your e-mail address: (Optional - this will NOT appear on the web page)

 
Comments:


 

Enter this text into the box below

Confirmation Code (required*) (what is this?)
(* enter the 5-digit number from the image above into the box below)

 
  

Note: Click the Submit button only once, then wait about 10 secs for a response
Do remember to enter the 5-digit confirmation code from the image, before clicking on Submit.

(This page last modified on Mar. 19, 2010)

 
Back to Top
Back to The Basement Computing Page
Back to The Basement

Please send questions, complaints or comments to kochhar@physiology.wisc.edu