[OpsMgr 2007 R2][OpsMgr 2012] System Center Data Access or Management Configuration services fail to start after applying KB2677070

Summary

After applying KB2677070 (http://support.microsoft.com/kb/2677070), the System Center Data Access service or System Center Management Configuration service may fail to start with a TimeOut error.

Direct link to Microsoft article

More Information

This issue occurs because the update changes the URLs used to contact Windows Update to download the trusted and untrusted CTLs. If the old URLs were hardcoded as exceptions in the firewall or proxy, the server running the Data Access service or the Management Configuration service will fail to download the new CTLs because it can't reach the updated web address.

The workaround for this is to unblock the updated URLs in the firewall or proxy or disable CRL checking for the Data Access service and Management Configuration service.

The updated URLs are:

http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab

(http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab)

http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab

(http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab)

Open the following file in a text editor:
- for the Data Access service: Microsoft.Mom.Sdk.ServiceHost.exe.config
- for the Management Configuration service: Microsoft.Mom.ConfigServiceHost.exe.config (in SM) or cshost.exe.config (in OM)

To disable CRL checking add the following line in the <runtime> section:

<generatePublisherEvidence enabled="false"/>

Below is an example of this tag being added for System Center 2012 Operations Manager:

 <runtime>
<generatePublisherEvidence enabled="false"/>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="Microsoft.EnterpriseManagement.HealthService" publicKeyToken="31bf3856ad364e35" />
          <publisherPolicy apply="no" />
          <bindingRedirect oldVersion="6.0.4900.0" newVersion="7.0.5000.0" />
        </dependentAssembly>
        <publisherPolicy apply="no" />
        <probing privatePath="" />
      </assemblyBinding>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="Microsoft.Mom.Common" publicKeyToken="31bf3856ad364e35" />
          <publisherPolicy apply="no" />
          <bindingRedirect oldVersion="6.0.4900.0" newVersion="7.0.5000.0" />
        </dependentAssembly>
        <publisherPolicy apply="no" />
        <probing privatePath="" />
      </assemblyBinding>
      <gcServer enabled="true"/>
    </runtime>


The next example shows the same parameter added in the configuration file for System Center Operations Manager 2007 R2:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <runtime>
<generatePublisherEvidence enabled="false"/>
        <gcServer enabled="true"/>
    </runtime>


The two*.config files can be found in the following directories:

-System Center Operations Manager 2007 R2: %ProgramFiles%\System Center Operations Manager 2007
-System Center Service Manager 2010: %ProgramFiles%\System Center Service Manager 2010
-System Center 2012 - Operations Manager: %ProgramFiles%\System Center 2012\Operations Manager\Server
-System Center 2012 - Service Manager: %ProgramFiles%\System Center 2012\Service Manager

Properties

Article ID: 2730040 - Last Review: October 2, 2012 - Revision: 3.0

Applies to

• Microsoft System Center 2012 Operations Manager
• Microsoft System Center Operations Manager 2007 R2
• Microsoft System Center 2012 Service Manager
• Microsoft System Center Service Manager 2010

Keywords:

kbtshoot KB2730040

This posting is provided "AS IS" with no warranties.

[OpsMgr 2007R2][OpsMgr 2012] The System Center Data Access Service fails to start after applying KB2677070

The System Center Data Access Service fails to start after applying KB2677070

Article ID: 2730040 - View products that this article applies to.

 

After applying KB2677070 (http://support.microsoft.com/kb/2677070), the System Center Data Access Service may fail to start with a Timeout error.

 

This issue occurs because the update changes the URLs used to contact Windows Update to download the trusted and untrusted CTLs. If the old URLs were hardcoded as exceptions in the firewall or proxy, the server running the Data Access Service will fail to download the new CTLs because it can't reach the updated web address.

The workaround for this is to ublock the updated URLs in the firewall or proxy or disable CRL checking for the Data Access Service.

The updated URLs are:

http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab

(http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab)

http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab

(http://ctldl.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab)

To disable CRL checking for the Data Access Service, open Microsoft.Mom.Sdk.ServiceHost.exe.config in a text editor and add the following line in the <runtime> section:

<generatePublisherEvidence enabled="false"/>

Below is an example of this tag being added for System Center 2012 Operations Manager:

 <runtime>
<generatePublisherEvidence enabled="false"/>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="Microsoft.EnterpriseManagement.HealthService" publicKeyToken="31bf3856ad364e35" />
          <publisherPolicy apply="no" />
          <bindingRedirect oldVersion="6.0.4900.0" newVersion="7.0.5000.0" />
        </dependentAssembly>
        <publisherPolicy apply="no" />
        <probing privatePath="" />
      </assemblyBinding>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="Microsoft.Mom.Common" publicKeyToken="31bf3856ad364e35" />
          <publisherPolicy apply="no" />
          <bindingRedirect oldVersion="6.0.4900.0" newVersion="7.0.5000.0" />
        </dependentAssembly>
        <publisherPolicy apply="no" />
        <probing privatePath="" />
      </assemblyBinding>
      <gcServer enabled="true"/>
    </runtime>


The next example shows the same parameter added in the configuration file for System Center Operations Manager 2007 R2:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <runtime>
<generatePublisherEvidence enabled="false"/>
        <gcServer enabled="true"/>
    </runtime>


Microsoft.Mom.Sdk.ServiceHost.exe.config is found in the following directories (where x is the drive on which Operations Manager is installed):

·         System Center Operations Manager 2007 R2: x:\Program Files\System Center Operations Manager 2007
·         System Center 2012 - Operations Manager: x:\Program Files\System Center 2012\Operations Manager\Server

Note This is a "FAST PUBLISH" article created directly from within the Microsoft support organization. The information contained herein is provided as-is in response to emerging issues. As a result of the speed in making it available, the materials may include typographical errors and may be revised at any time without notice. See Terms of Use

(http://go.microsoft.com/fwlink/?LinkId=151500)

for other considerations.

 

Article ID: 2730040 - Last Review: July 9, 2012 - Revision: 2.0

APPLIES TO

• Microsoft System Center 2012 Operations Manager
• Microsoft System Center Operations Manager 2007 R2

Keywords:

kbtshoot KB2730040

This posting is provided "AS IS" with no warranties.

[OpsMgr 2007] Timeout running Remove-DisabledMonitoringObject cmdlet in System Center Operations Manager

I've read a lot and seen a lot of cases like mine - I mean remove-disabledmonitoringobject cmdlet get the error "The requested operation timed out" but it seems that nobody has been lucky to found how to fix the issue. We have openned a Micorsoft case on this issue a lot of monthes ago and since the begining of the week, we are now able to run the cmdlet without any error.

This resolution given by Microsoft for an openned case will not be part of the CU6.
The resolution has been found too late !

First, here is our configuration - approximatly 3000 agents in CU3. We don't have upgraded to CU4 and we soon upgrade to CU5. SQL 2008 for the DBs.

The first thinking of why the cmdlet is timing out was the number of overrides was too much for the SDK to process before the thirty minute WCF(Windows Connection Framework) timeout occurs.

Within our production environment there are approximately 140000 DiscoverySources which need analyse be the cmdlet to know if the associated discovered types need to be removed or not. We have a pre-production environnemnt with less number of agents and only 40000 DiscoverySources on wich the cmdlet is well working.

To reduce the number of Discovery sources we have analysed all the MP we have and it appeared that OCS MP was responsible for almost 40% of the hugh number of DiscoverySources. The OCS MP has nineteen discoveries targeted at Windows Server Computer class enabled by default. On each discoveries we have an override on a group to disable disovery for the group members. A discoverysource entry is created for each discovery-to-target-entity mapping.
We have also :  19 * ~3000 agents = 57000 discoverysources just for OCS MP.
When overrides are done one theses discoveries, the enabled states must calculated for all discoverysources !

I've worked to remove some overrides and also to reduce the enabled state calculation for the DiscoverySources but the cmdlet was always timed out.

The next way to fix the issue was to let Microsoft have an other review of the code and SQL involved to see if they can make some efficiencies in the way they do this. I've also been asked to run 2 queries on the SCOM Database :
I’ve also run the following queries :

1. SELECT COUNT (Distinct [DiscoverySource].[DiscoverySourceId])
2. FROM dbo.DiscoverySource
3. INNER JOIN dbo.ModuleOverride ON ModuleOverride.ParentId = DiscoverySource.DiscoveryRuleId
4. AND ModuleOverride.OverrideableParameterId = dbo.fn_MPObjectId(NULL, NULL, N'Enabled')
5. AND (ParentType = 'Discovery' OR ParentType = 'Rule')
6. join DiscoverySourceToTypedManagedEntity dstme
7. on discoverysource.DiscoverySourceId = dstme.DiscoverySourceId
8. WHERE DiscoverySource.IsDeleted = 0
9. AND ModuleOverride.Value = 'false'

1. SELECT COUNT (Distinct [DiscoverySource].[DiscoverySourceId])
2. FROM dbo.DiscoverySource
3. INNER JOIN dbo.ModuleOverride ON ModuleOverride.ParentId = DiscoverySource.DiscoveryRuleId
4. AND ModuleOverride.OverrideableParameterId = dbo.fn_MPObjectId(NULL, NULL, N'Enabled')
5. AND (ParentType = 'Discovery' OR ParentType = 'Rule')
6. join DiscoverySourceToTypedManagedEntity dstme
7. on discoverysource.DiscoverySourceId = dstme.DiscoverySourceId
8. WHERE DiscoverySource.IsDeleted = 0

Given the numbers returned by the queries Microsoft support suspects the step below will allow the cmdlet to complete and await your results.

Here is also what I've been asked to do :

• Run the SQL against the OperationsManager DB.

1. DECLARE @querydef XML
2. SET @querydef =
3. N'<QueryDefinitions xmlns="urn:DataAccess" xmlns:dal="urn:DataAccess" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:DataAccess QueryDefinition.xsd">
4. <QueryDefinition>
5.   <Name>DiscoverySourcesEligibleForDeletionDueToOverrides</Name>
6.   <ObjectName>DiscoverySourcesEligibleForDeletionDueToOverrides</ObjectName>
7.   <UsedBy Component="Sdk" />
8.   <Description>Selects discovery sources that *may* be invalid due to applied overrides.</Description>
9.   <DataObject xsi:type="SelectType">
10.     <Column>
11.       <Name>DiscoverySourceId</Name>
12.       <Source>DiscoverySource</Source>
13.       <Type>uniqueidentifier</Type>
14.     </Column>
15.     <Column>
16.       <Name>DiscoverySourceType</Name>
17.       <Source>DiscoverySource</Source>
18.       <Type>tinyint</Type>
19.       <EnumType>Microsoft.EnterpriseManagement.Mom.Modules.DataItems.Discovery.DiscoverySourceType</EnumType>
20.       <EnumLeastValue>Rule</EnumLeastValue>
21.       <EnumGreatestValue>ConfigService</EnumGreatestValue>
22.     </Column>
23.     <Column>
24.       <Name>DiscoveryRuleId</Name>
25.       <Source>DiscoverySource</Source>
26.       <Type>uniqueidentifier</Type>
27.     </Column>
28.     <Column>
29.       <Name>BoundManagedEntityId</Name>
30.       <Source>DiscoverySource</Source>
31.       <Type>uniqueidentifier</Type>
32.     </Column>
33.     <Argument>Distinct</Argument>
34.     <Source>
35.       <Table>
36.         <Name>DiscoverySource</Name>
37.         <Owner>dbo</Owner>
38.         <Type>Table</Type>
39.       </Table>
40.       <Join>
41.         <Type>Inner</Type>
42.         <Table>
43.           <Name>ModuleOverride</Name>
44.           <Owner>dbo</Owner>
45.           <Type>Table</Type>
46.         </Table>
47.         <JoinCondition>ModuleOverride.ParentId = DiscoverySource.DiscoveryRuleId AND ModuleOverride.OverrideableParameterId = dbo.fn_MPObjectId(NULL, NULL, N''Enabled'') AND (ParentType = ''Discovery'' OR ParentType = ''Rule'')</JoinCondition>
48.       </Join>
49.       <Join>
50.         <Type>Inner</Type>
51.         <Table>
52.           <Name>DiscoverySourceToTypedManagedEntity</Name>
53.           <Owner>dbo</Owner>
54.           <Type>Table</Type>
55.         </Table>
56.         <JoinCondition> discoverysource.DiscoverySourceId = DiscoverySourceToTypedManagedEntity.DiscoverySourceId</JoinCondition>
57.       </Join>
58.     </Source>
59.     <Conditional>
60.       <Condition>
61.         <Expression>DiscoverySource.IsDeleted = 0 AND ModuleOverride.Value = ''false''</Expression>
62.       </Condition>
63.     </Conditional>
64.   </DataObject>
65. </QueryDefinition>
66. </QueryDefinitions>'
67. INSERT INTO dbo.[DataAccessLayerSetting]([SettingType], [SettingData]) VALUES (0, @querydef)

The impact of this change is when the cmdlet is run, it will now use the SQL query inserted in the Data.AccessLayerSetting table instead of the Original SQL query which is compiled into one dll. This table allows us to override the in-built queries, as it is read on OMSDK service restart.Coming back to the origicnal SQL is very easy, just remove the entry in the Data.AccessLayerSetting table and restart the SDK.

• Restart the OpsMgr SDK service.
• Run the remove-disabledmonitoringobject cmdlet and report back on the success or failure of it.

I use to launch the cmdlet like this (in a short PS1):

1. get-managementserver | select ManagementGroup -unique
2. get-date
3. remove-disabledmonitoringobject
4. get-date

That permit to show in the same few line the OpsMgr group, the dates before and after the cmdlet has run.

 Unfortunatly the 2 first time I've launched the remove-disabledmonitoringobject cmdlet, it went to a new error :

1. >get-date
2. Monday, November 28, 2011 8:27:16 AM
3. PS Monitoring:\
4. >remove-disabledmonitoringobject
5. Remove-DisabledMonitoringObject : Microsoft.EnterpriseManagement.Common.DiscoveryDataFromRuleTargetedToDeletedMonitoringObjectException: Discovery data has been received from a rule targeted at a non-existent monitoring object id.
6. MonitoringObjectId: c1537246-ec53-cc7c-b45f-aed87f06bc7f
7. RuleId: 66b6d462-535f-cab6-eb14-b24fc79dfb75
8.    at Microsoft.EnterpriseManagement.DataAbstractionLayer.InstanceSpaceOperations.DeleteDisabledDiscoverySources()
9.    at Microsoft.EnterpriseManagement.ManagementGroup.DeleteDisabledMonitoringObjects()
10.    at Microsoft.EnterpriseManagement.OperationsManager.ClientShell.RemoveDisabledMonitoringObjectCmdlet.ProcessRecord()
11. At line:1 char:32
12. + remove-disabledmonitoringobject <<<<
13.     + CategoryInfo          : InvalidOperation: (Microsoft.Enter...ingObjectCmdlet   :RemoveDisabledMonitoringObjectCmdlet) [Remove-DisabledMonitoringObject], Disc
14.   overyDataFr...ObjectException    + FullyQualifiedErrorId : ExecutionError,Microsoft.EnterpriseManagement.Operat
15.    ionsManager.ClientShell.RemoveDisabledMonitoringObjectCmdlet
16. PS Monitoring:\
17. >get-date
18. Monday, November 28, 2011 8:48:55 AM

I've also report this again to the microsoft support and wait any answer. That particular seems to be known and the cmdlet completed without error on the second run. In my case, the second run of the cmdlet found a different object which caused again the error and as in the first run it set the original object’s IsDeleted property to 1. It appears that the cmdlet attempts remove some objects twice, and it's failing on the second attempt.

I've launched a third time the cmdlet and it ended in success ! 

This posting is provided "AS IS" with no warranties.

This posting is provided "AS IS" with no warranties.