BizTalk Maintenance Jobs in SQL Server

I recently came across a situation where I had to identify bottlenecks on a BizTalk server which was performing like a snail. I have written a blog post about this here.

However, I also realised the importance of various BizTalk jobs in SQL Server during this process. I would like to share with you what I learnt in relation to this and how one should keep an eye on these sql jobs as they play a pivotal role in highly (& poorly) performing BizTalk setups.

There are about 12 sql jobs in BT 2009 and 13 jobs in BT 2010 which play their own little role when it comes to ensuring good BizTalk performance.

BizTalk Jobs

The DTA purge job is disabled by default and can be enabled if the tracking database gets too big.  The messagebox cleanup job is also disabled by default as it is triggered by the ManageRefCountLog job as needed.

It is very important to note that if these jobs dont run on a timely manner especially when the tracking and spool tables get bigger than the set throttling limit, BizTalk starts throttling. The moment BizTalk starts throttling, performance starts to degrade.

Lets look at what each job is meant to do (referenced from MS Support):

Job name Description Enabled by default
Backup BizTalk Server Consists of three steps. Step 1 performs full database backups of the BizTalk Server databases. Step 2 backs up the BizTalk Server database logs. Step 3 specifies for how long the backup history is kept. No
CleanupBTFExpiredEntriesJob_BizTalkMgmtDb Deletes expired BizTalk Framework entries from the BizTalk Management (BizTalkMgmtDb) database. Yes
DTA Purge and Archive Automates the archiving of tracked messages and the purging of the BizTalk Tracking database to maintain a healthy system and to keep the tracking data archived for future use.Note On BizTalk Server 2004, this job is created after you install BizTalk Server 2004 Service Pack 2. No
MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb Detects when a BizTalk Server host instance (BTSNTSvc.exe) has stopped responding. The job then releases the work from the host instance so a different host instance can finish the tasks. Yes
MessageBox_Message_Cleanup_BizTalkMsgBoxDb Removes all messages that are not referenced by any subscribers in the BizTalkMsgBoxDb database tables.Important This job is also started by the MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDbjob. Therefore, we recommend that you disable this job.Note On BizTalk Server 2004, this job is enabled by default. Therefore, we recommend that you disable this job. No
MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb Manages the reference count logs for messages and determines when a message is no longer referenced by a subscriber.This job runs in an infinite loop and deletes the entries from the two individual message reference count logs. This job also calls the MessageBox_Message_Cleanup_BizTalkMsgBoxDbjob.Important At first, the MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb job status icon displays a status of Success. However, there will be no corresponding success entry in the job history. If one of the jobs in the MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb job fails, a failure entry appears in the job history and the status icon displays a status of Failure. The job will always display a status of Failure after the first failure. To verify that the other BizTalk Server SQL Server Agent jobs run correctly, check the status of the other BizTalk Server SQL Server Agent jobs.

Note On BizTalk Server 2004, this job is created after you install BizTalk Server 2004 Service Pack 2.

MessageBox_Parts_Cleanup_BizTalkMsgBoxDb Removes all message parts that are no longer referenced by a message in the BizTalkMsgBoxDb database tables. All messages are composed of one or more message parts that contain the message data. Yes
MessageBox_UpdateStats_BizTalkMsgBoxDb Updates the statistics for the BizTalkMsgBoxDb database.Note This job does not exist on BizTalk Server 2004. Yes
Monitor BizTalk Server Scans for any known issues with the BizTalkMgmtDb, BizTalkMsgBoxDb, and BizTalkDTADb databases. This includes orphaned instances.Note This job is created on BizTalk Server 2010. Yes
Operations_OperateOnInstances_OnMaster_BizTalkMsgBoxDb Used for multiple BizTalkMsgBoxDb database deployment. It asynchronously performs operational actions. For example, it asynchronously performs bulk terminates on the master BizTalkMsgBoxDb database after those changes are applied to the subordinate BizTalkMsgBoxDb database. Note This job does not exist on BizTalk Server 2004.  Yes
PurgeSubscriptionsJob_BizTalkMsgBoxDb Purges unused subscription predicates from the BizTalkMsgBoxDb database. Yes
Rules_Database_Cleanup_BizTalkRuleEngineDb Purges old audit data from the Rule Engine (BizTalkRuleEngineDb) database every 90 days. This job also purges old history data (deploy/undeploy notifications) from the Rule Engine (BizTalkRuleEngineDb) database every 3 days.Note This job is created on BizTalk Server 2009. Yes
TrackedMessages_Copy_BizTalkMsgBoxDb Copies the message bodies of tracked messages from the BizTalkMsgBoxDb database to the Tracking (BizTalkDTADb) database. Yes
TrackingSpool_Cleanup_BizTalkMsgBoxDb Purges inactive tracking spool tables to free database space.Note This job exists only on BizTalk Server 2004.