SharePoint and the case of the broken help

Greetings fellow SharePoint nugget seekers!

Problem

Unfortunately, help seems to be broken… there aren’t any help collections in the current language for the site you’re using.

Let me guess. You’ve recently gone live with a SharePoint environment and now a user has reported that clicking the help button displays the above message.

Let me ask. Did you by any chance rid your farm databases of those long unwieldy GUIDs in their names? To be specific, I am referring to the content database that hosts Central Administration. Perhaps you followed the steps in one of these articles (or similar ones, there are many out there).

• Getting the GUID out of the AdminContent database in SharePoint 2010 – Todd Klindt’s SharePoint
• Renaming the SharePoint Admin Content Database to Follow SQL Server Naming Conventions
• Surya’s Blog: Rename Admin Content database in SharePoint 2013

I did. I wanted nice clean database names and yes I followed the steps in these articles. And then I got this error message being reported. It is only a mildly annoying side-effect, but it’s that impression on users when they are actually seeking assistance and find this error message behind the help button that I want to avoid. After all, I followed the guidance in those articles to make the environment cleaner, not make more problems.

Solution

Provide SPDataAccess permissions to the central administration content database for all web application pool identities. There is no need for any iisreset or system outage.

Don’t like this? Have a look at my list of alternative approaches.

Why is that the solution?

The approach of blog posts like the ones I linked to above is to create a brand new content database to host the Central Administration site collection. However the content database that gets created is a generic SharePoint content database while the Central Administration content database is in fact a bit special! There are a few extra bits and pieces that it is given, namely the database security roles WSS_Content_Application_Pools and SharePoint_Shell_Access.

When SharePoint is trying to render the help window, the web application pool identity requires some level of access to the central administration content database. I have experimented with the security roles that are available and found that SPDataAccess provides enough access to resolve the broken help message. You can always grant db_owner to the application pool account, however if you are trying to stay with least privilege security practices this may be as close as you can get without completely rebuilding the environment.

Prove the solution works

I have built a SharePoint 2013 environment and patched it to August 2015 cumulative update (or 15.0.4745.1000 if you were wondering).

The environment has a single web application on port 80 and a single site collection provisioned at the root of the web application.

The farm administrator account, web application pool identity, and site collection owner are three distinct domain accounts.

The SQL instance currently has databases for

• Central Administration content database (with GUID)
• Configuration database
• Web application content database

When opening help on the provisioned site collection website, the popup dialog displays successfully.

From this point, I executed some PowerShell similar to the steps described in the articles linked to above. This created the new content database and migrated the Central Administration site collection.

Here are the databases in the instance now including the new database created without GUID.

At this point, attempting to open the help dialog displays the error message.

The attempt to open the help dialog box is logged in the web server application event log as ID 3760. SharePoint also logs an error in the ULS described as SQLClient.SQLException. A full stack trace is listed at the end of this blog post.

Now to follow the solution, provide the web application pool identity account access to the newly created content database with security role SPDataAccess.

Attempting to open the help dialog is now successful.

Here is a side-by-side look at two databases, the database created when provisioning the farm with GUID and the manually created database without GUID.

Notice the GUID database contains security roles for SharePoint_Shell_Access and WSS_Content_Application_Pools

Alternative approaches

psconfig
There is a command to create, connect or disconnect a server from the server farm. This includes a parameter to specify the central administration content database.
i.e. psconfig.exe -cmd configdb … -admincontentdatabase
Unfortunately this does not provide functionality to rename databases on farms that already exist.

build the farm with powershell
The cmdlet New-SPConfigurationDatabase has a parameter to specify the name of the Administration content database. But let’s face it, the reason we are in this situation is because we chose the wizard interface. It’s not really an option in the scope of this article to consider a farm rebuild.

rename the database
Wow did I really put this here as an alternative? In what universe can you rename a database and expect the application to keep working? Sorry there is no option to rename the central administration content database from any SharePoint interface. This is listed here just because I considered it as an option to see if it could solve the problem.

backup database, unmount, restore database renamed
No this is also not supported by SharePoint… sure it might be a possibility for other content databases, but since we are dealing with central administration just forget it.

Other “Help” error messages

You might come across other symptoms of this problem in other places. I’ve listed a few places that I saw the error message generated.

Performing a search

Editing a view

Appendix

Details of application event log

SQL Database 'SharePoint_AdminContent' on SQL Server instance 'SQL INSTANCE' not found. Additional error information from SQL Server is included below.
Cannot open database "SharePoint_AdminContent" requested by the login. The login failed. Login failed for user 'DOMAIN\WebAppPool'.

Details of ULS log stack trace
08/28/2015 14:47:10.19 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Monitoring nasq Medium Entering monitored scope (Request (GET:http://SERVER:80/_layouts/15/help.aspx?Lcid=1033&Key=WSSEndUser&ShowNav=true)). Parent No
08/28/2015 14:47:10.19 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Logging Correlation Data xmnv Medium Name=Request (GET:http://SERVER:80/_layouts/15/help.aspx?Lcid=1033&Key=WSSEndUser&ShowNav=true) e54c289d-3657-a014-0000-08d685ddb864
08/28/2015 14:47:10.39 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Database 880i High System.Data.SqlClient.SqlException (0x80131904): Cannot open database "SharePoint_AdminContent" requested by the login. The login failed. Login failed for user 'DOMAIN\WebAppPool'. at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.SharePoint.Utilities.SqlSession.OpenConnection() ClientConnectionId:df7f709f-7560-443b-95bc-8832ed908466 Error Number:4060,State:1,Class:11 e54c289d-3657-a014-0000-08d685ddb864
08/28/2015 14:47:10.39 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Database 880k High at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command, CommandBehavior behavior, SqlQueryData monitoringData, Boolean retryForDeadLock) at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command, Boolean retryForDeadLock) at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command) at Microsoft.SharePoint.Upgrade.SPDatabaseSequence.GetVersion(SPDatabase database, Guid id, Version defaultVersion, SqlSession session, SPDatabaseSequence sequence) at Microsoft.SharePoint.Upgrade.SPDatabaseSequence.GetVersion(Guid id, Version defaultVersion) at Microsoft.SharePoint.Upgrade.SPDatabaseSequence.get_SchemaVersion() at Microsoft.SharePoint.Upgrade.SPSequence.get_IsBackwardsCompatible() at Microsoft.SharePoint.Upgrade.SPUpgradeSession.IsBackwardsCompatible(Object o, Boolean bRecurse) at Microsoft.SharePoint.Administration.SPPersistedUpgradableObject.get_IsBackwardsCompatible() at Microsoft.SharePoint.Administration.SPPersistedUpgradableObject.ValidateBackwardsCompatibility() at Microsoft.SharePoint.SPSite.PreinitializeServer(SPRequest request) at Microsoft.SharePoint.SPWeb.InitializeSPRequest() at Microsoft.SharePoint.SPWeb.EnsureSPRequest() at Microsoft.SharePoint.SPWeb.InitWebPublic() at Microsoft.SharePoint.Help.HelpContextManager.get_ProductHelpLibraryUrl() at Microsoft.SharePoint.Help.HelpContextManager.GetWebHelpUrl(SPWeb web) at Microsoft.SharePoint.Help.HelpContextManager.get_ContextWebHelpUrl() at Microsoft.SharePoint.ApplicationPages.HelpPage.GetHelpItemAssetKey(SPQueryProvider siteHelpQueryProvider, SPQueryProvider productHelpQueryProvider) at Microsoft.SharePoint.ApplicationPages.HelpPage.b__0() at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass5.b__3() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode) at Microsoft.SharePoint.ApplicationPages.HelpPage.OnLoad(EventArgs e) at System.Web.UI.Control.LoadRecursive() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) at System.Web.UI.Page.ProcessRequest() at System.Web.UI.Page.ProcessRequest(HttpContext context) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error) at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb) at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus) at System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr pHandler, RequestNotificationStatus& notificationStatus) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr rootedObjectsPointer, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags) e54c289d-3657-a014-0000-08d685ddb864
08/28/2015 14:47:10.39 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Database 880j High SqlError: 'Cannot open database "SharePoint_AdminContent" requested by the login. The login failed.' Source: '.Net SqlClient Data Provider' Number: 4060 State: 1 Class: 11 Procedure: '' LineNumber: 65536 Server: 'SQL INSTANCE' e54c289d-3657-a014-0000-08d685ddb864
08/28/2015 14:47:10.39 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Database 880j High SqlError: 'Login failed for user 'DOMAIN\WebAppPool'.' Source: '.Net SqlClient Data Provider' Number: 18456 State: 1 Class: 14 Procedure: '' LineNumber: 65536 Server: 'SQL INSTANCE' e54c289d-3657-a014-0000-08d685ddb864
08/28/2015 14:47:10.39 w3wp.exe (0x1594) 0x2978 SharePoint Foundation Database 3760 Critical SQL Database 'SharePoint_AdminContent' on SQL Server instance 'SQL INSTANCE' not found. Additional error information from SQL Server is included below. Cannot open database "SharePoint_AdminContent" requested by the login. The login failed. Login failed for user 'DOMAIN\WebAppPool'. e54c289d-3657-a014-0000-08d685ddb864

SharePoint 2010 crawl error with Kerberos constrained delegation KDC_ERR_BADOPTION

I have a SharePoint 2010 environment set up to use SQL Server Reporting Services in SharePoint integrated mode.

I have followed the guide from Microsoft to configuring SharePoint Kerberos (http://www.microsoft.com/en-au/download/details.aspx?id=23176)

I have enabled Kerberos event logging (http://support.microsoft.com/kb/262177)

I am now getting errors in the System log file every hour per my search incremental crawl schedule.
A Kerberos Error Message was received:
on logon session
Client Time:
Server Time: 22:56:15.0000 4/1/2013 Z
Error Code: 0xd KDC_ERR_BADOPTION
Extended Error: 0xc0000272 KLIN(0)
Client Realm:
Client Name:
Server Realm: <>
Server Name: <>@<>
Target Name: <>@<>@<>
Error Text:
File: 9
Line: f09
Error Data is in record data.

A bit of investigation later, I found this on the Active Directory Team blog http://blogs.technet.com/b/askds/archive/2012/07/27/kerberos-errors-in-network-captures.aspx

If the domain controller returns KDC_ERR_BADOPTION, it means that one of the KrbFlags set in the KdcOptions is not allowed… The most common scenario is a request for a delegated ticket (unconstrained or constrained delegation)… There are several reasons for rejection…

The request was for a constrained delegation ticket to itself (constrained delegation is designed to allow a middle tier service to request a ticket to a back end service on behalf on another user, not on behalf of itself).

Now hang on, what was it that the Microsoft guide said? (http://technet.microsoft.com/en-us/library/gg502602(v=office.14).aspx)

Note:
It may seem redundant to configure delegation from a service to itself, such as the portal service account delegating to the portal service application, but this is required in scenarios where you have multiple servers running the service. This is to address the scenario where one server may need to delegate to another server running the same service; for instance a WFE processing a request with a RSS viewer which uses the local web application as the data source. Depending on farm topology and configuration there is a possibility that the RSS request may be serviced by a different server which would require delegation to work correctly.

So if that particular scenario provided by Microsoft applies to your situation, go ahead and configure constrained delegation for the service account to itself, but expect to see these errors in the System log.

My conclusion from all this? It seems that Kerberos configuration is different for everyone, almost impossible to learn, and that the SQL stack of products needs to be claims aware as soon as possible.

Office Web Applications on SharePoint 2010 Troubleshooting

A user reported an issue on our environment attempting to open a Word document. The interface was presenting a simple error message and returning them to the SharePoint website.

Word Web App cannot open this document for viewing because of an unexpected error. To view this document, open it in Microsoft Word.

Diligent observers would note a slight difference in the button options available between these two error message dialogs. I’m not sure what triggers one or the other, other than the first dialog is displayed at first encounter of the error, but they are very similar and seem interchangeable.

There were other error messages found along the way.

Word Web App cannot open this document because the Office Web Apps Feature isn’t activated on this site collection. Try opening the file in Microsoft Word.

Word Web App cannot open this document for editing because it is currently being edited by another user.

This post is to document the steps I followed in troubleshooting because while I found lots of information available on Microsoft KB articles and around various forums, nothing seemed to be of much use to my particular situation.

I firstly reproduced the issue by opening the document (feature to open documents in Office Web Apps by default is enabled) and checked out the ULS. Here is what I found:

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C SharePoint Foundation Logging Correlation Data xmnv Medium Name=Request (GET:https: //SERVER/sites/SITECOLLECTION/_vti_bin/docdatahandler.ashx?d=F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m&z=85BE98FB-0637-46E4-BF39-DE1591AED6A02&type=sl&ui=en-US) 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C SharePoint Foundation Logging Correlation Data xmnv Medium Site=/sites/azuurifd 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture vjyu Medium Librarian.BeginGetItem(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture vkau Medium ViewStore.BeginGetItem(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture e9dk Medium ViewStore.ReadMainCache(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.89 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture el9d Medium FEMemoryCache.FReadFromCacheLine: d=F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m&z=85BE98FB-0637-46E4-BF39-DE1591AED6A02, 00000000-0000-0000-0000-000000000101, length 10240, offset 0 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.91 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture vkaw Medium ViewStore.SetCompleted(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) - status = DocumentOutputNotFound 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.91 w3wp.exe (0x1190) 0x361C Office Web Apps Office Viewing Architecture b4vw Medium RequestDispatcher: dispatching request for document F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m to http: //MACHINENAME:32843/4dccdb5d5def497e84a0db1b4016b370/Conversion.svc. 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:37.92 w3wp.exe (0x1190) 0x361C SharePoint Foundation Topology e5mc Medium WcfSendRequest: RemoteAddress: 'http://MACHINENAME:32843/4dccdb5d5def497e84a0db1b4016b370/Conversion.svc' Channel: 'Microsoft.Office.Web.Conversion.Framework.Remoting.IConvertingService' Action: 'http: //tempuri.org/IConvertingService/ProcessRequest' MessageId: 'urn:uuid:0a64f724-f9b6-4c67-b43c-f989c4f7ea39' 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:38.08 w3wp.exe (0x1190) 0x2840 Office Web Apps Office Viewing Architecture dg2s Medium RequestDispatcher: response received for document F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m from http: //MACHINENAME:32843/4dccdb5d5def497e84a0db1b4016b370/Conversion.svc. 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:38.08 w3wp.exe (0x1190) 0x2840 Office Web Apps Office Viewing Architecture b4vn Medium Conversion response received: CantFindSourceDocument 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:38.08 w3wp.exe (0x1190) 0x2840 Office Web Apps Office Viewing Architecture vjza Medium Librarian.OnConversionResponseReceived(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) - Response=CantFindSourceDocument, Data=NO 460f8992-a2e7-4431-9e8f-607c1ac4953d

11/27/2012 11:44:38.08 w3wp.exe (0x1190) 0x2840 Office Web Apps Office Viewing Architecture vjzo Medium Librarian.SetCompleted(F2bcc7d18ae564960ba19f23bd5314f50m048c6be7923b44c9aeedd698d645aaa1m85be98fb063746e4bf39de1591aed6a0m, Silverlight, docdata.xml) - status = ConversionError 460f8992-a2e7-4431-9e8f-607c1ac4953d

My observations

• The Web server reports that it can’t find the source document
• The Web server is making a call to another machine in the farm hosting a Word Web App service instance. You may not always see this call due to various caching going on.

Meanwhile, another Word document in the same site collection/site/library/folder was rendering successfully in Word Web App.

There is a Microsoft KB article with some generic advice here http://support.microsoft.com/kb/2028346 such as check your service account permissions and ensure the service application proxy is associated with the web application. I could rule these things out since the Word Web App was successfully working for at least one document. So on to further research.

The first interesting blog post I found http://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/23/Office-Web-Apps-_2D00_-Word-Viewing-Service-Error.aspx suggested the “Edit in Browser” functionality was separate to the “View in Browser” functionality. I gave it a go and presto the document was successfully rendering in the browser.

This was actually the first time I saw the contents of the document, and straightaway noted that it contained some unusual characters that Word Web App had rendered as “[Soft Break]”.

Since this seemed a case of “the odd one out”, I made a copy of the document (being careful to retain all metadata) and edit it so that only the Soft Break characters were removed. Testing showed that Word Web App was successfully rendering the document.

There was also an article I read during my research that suggested Soft Break characters had limited success in Office Web Apps (which I cannot find again despite my best googling – it was a Google Docs VS Office 365 kind of article so the author may have had some bias one way or the other).

The case seemed to be closed.

The user then reported an issue with the PowerPoint file on his website not rendering and displaying an error message.

PowerPoint Web App cannot open this presentation for viewing because of an unexpected error. To view this presentation, open it in Microsoft PowerPoint.

and

PowerPoint Web App encountered an error. Please try again.

I looked at the file, and the first thing I did was set about searching for odd characters. I uploaded various copies of the file containing subsets of the original’s slides to try and deduce where any odd characters may exist, but the results were mixed. A document containing one subset of the slides would successfully render, but another would not. I tried a number of combinations eventually giving up on finding any pattern or odd character.

Next I uploaded a brand new empty PowerPoint 2010 file (default master slide), which the PowerPoint Web App was unable to render, showing the error message.

I then tried uploading the user’s PowerPoint file to another farm and found PowerPoint Web App rendering the document just fine. Just for kicks, I also uploaded the user’s Word file and found that it too was working successfully, even with Soft Break characters.

Now the cause seemed to be pointing to the Office Web Apps on the offending farm. There also seemed to be some intermittent behaviour since there was at least one Word document that would successfully render for viewing in Word Web App. I set about investigating this by first off restarting the service instances on each machine in the farm and then retesting the documents – no good. I re-provisioned the service applications and retested – no good.

There is a Microsoft TechNet article that outlines some generic advice how to troubleshoot Office Web Apps here http://technet.microsoft.com/en-us/library/hh269604(v=office.14).aspx. Being diligent and wanting to rule out anything obvious this article covered, I confirmed the application pool identity was granted db_owner on the content databases via GrantAccessToProcessIdentity (http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.spwebapplication.grantaccesstoprocessidentity.aspx) then retested – and still no good.

Small aside, a new error message started to display for Word Web App’s “Edit in Browser” function after making these changes.

Word Web App cannot open this document because the Office Web Apps Feature isn’t activated on this site collection. Try opening the file in Microsoft Word.

Fortunately this was resolved with a simple iisreset.

More research followed – I read two blogs.

http://heidibrebels.be/blog/archives/2011/office-web-apps-error-sharepoint
This was the most comprehensive article I found approaching Office Web Apps errors on SharePoint 2010… at least one that went beyond the generic and basic information repeated in numerous other articles. It even contained the same ULS entries as I found, although I’m not too sure about the logic that suggested installing Silverlight.

http://blogs.technet.com/b/wbaer/archive/2010/09/01/the-office-web-applications-cache.aspx
Bill Baer, the Godfather of SharePoint infrastructure, comes to the rescue. Bill does not provide the answer to any Office Web Apps issues, but does enlighten us on the actual mechanics behind Office Web Apps cache. Using this knowledge, I was able to dig in to the Office Web Apps infrastructure and resolve the issue. Here is what I did:

• Check the cache creation timer job – ok.
• Check the cache site collection existed – ok.
• I did not need to isolate the office web application cache to a separate content database.
• Clear the content in the site collection cache – ok.

Now when I was re-testing the Word Web App rendering a document for viewing, I could see that nothing was being created in the site collection cache.

• Check the server file system cache

The interesting piece of information here was the location of the server file system cache.

• Word Web App server file system cache is located at C:\Windows\Temp\waccache\SERVICEAPPLICATIONGUID\SERVICEACCOUNT\
• PowerPoint Web App server file system cache is located at C:\Windows\Temp\powerpointcache\UNIDENTIFIEDGUID\viewing\SERVICEACCOUNT

Now when I was re-testing, I saw that I needed to be on the server that was being called for “Conversion.svc” i.e. the machine that was being used for the service instance. Here I could see the cache document being created, and ‘disappearing’ a few minutes later. I would assume that a timer job is picking this up and moving it to the site collection cache.

Finally, per Bill’s post

• I cleared the server file system cache – ok.
• Perform IIS reset on all servers in the farm – ok.

On retesting, the issues were all gone. I hope to clarify this step further and refine what is actually required, but a resolution is a resolution.

EDIT – I believe the only reset required is the application pool for the Office Web App on the server hosting the service instance.

IIS web application pool configuration

I saw a cool little command today so I will just post it here for my memory.

%windir%\system32\inetsrv\appcmd list apppool
%windir%\system32\inetsrv\appcmd list apppool APPPOOLNAMEHERE /text:*

Doing this allowed me to see the long forgotten password of the service accounts used to install systems by people long departed. There are lots of other cool bits of info there as well.

Oh, this was on Windows Server 2008 R2.

PerformancePoint, Kerberos, Claims To Windows Token Service, and unhelpful error messages

A user complained that PerformancePoint was not configured correctly.

“See? SharePoint told me so!”

You check the logs and find this on the application server running the PerformancePoint service.

And corresponding in the SharePoint ULS log on the same server.

The user "DOMAIN\User" attempted to access an item in the following location: http: //SharePointServer/Data Connections for PerformancePoint/1_.000

Verify that the location exists and that the user has the "Read Items" permission.

Exception details: Microsoft.PerformancePoint.Scorecards.BpmException: The item 'http://SharePointServer/Data Connections for PerformancePoint/DataSource.rsds' is not a valid PerformancePoint Data Source. Check to make sure the content type of this item is correct.

However you can also find things related to Claims to Windows Token Service in ULS log.

SPSecurityContext.WindowsIdentity: Could not retrieve a valid windows identity for NTName='DOMAIN\User', UPN='User@DOMAIN'. UPN is required when Kerberos constrained delegation is used. Exception: System.ArgumentException: Token cannot be zero.

The following data source cannot be used because PerformancePoint Services is not configured correctly.

Data source location: http: //SharePointServer/Data Connections for PerformancePoint/5_.000 Data source name: New Data Source

Monitoring Service was unable to retrieve a Windows identity for "DOMAIN\User".

Verify that the web application authentication provider in SharePoint Central Administration is the default windows Negotiate or Kerberos provider.

If the user does not have a valid active directory account the data source will need to be configured to use the unattended service account for the user to access this data.

The best hint at the gremlin in the system is the error “Token cannot be zero” from the Claims to Windows Token service. The SharePoint system attempts to change the claims token to a windows token, but the service is just returning null. Basically this is a permissions error for the Claims to Windows Token Service identity.

If you read the SharePoint 2010 Kerberos Guide, particularly the chapter on configuring PerformancePoint, you would understand that there is a relationship between PerformancePoint and Claims to Windows Token Service. Too bad it doesn’t tell you how to fight system gremlins, but not to worry… that’s why I’m here.

The domain account you are using needs a few things to be able to successfully run the Claims to Windows Token service.

• Local administrator on the application server running PerformancePoint

And in Local Security Policy (secpol.msc)

• Act as part of the operating system
• Impersonate a client after authentication
• Log on as a service (the service account will probably already be here if it is running a service)

I found two gotchas in this particular environment.

• Windows group policy removed the Claims to Windows Token service account from local administrator. Contact your friendly Windows/Active Directory administrator for assistance on this.
• After the permissions are applied the Claims to Windows Token service account needs to be restarted in order to apply the permissions. If you do not restart the service, your user will still be getting the error message!

SharePoint Site Collection Administrators with PowerShell

Suppose you want to delete a user from Site Collection Administrator but not deal with the Owner and Secondary Owner properties business…

Mind you, even if you like to ignore ownership properties, there still needs to be at least one site collection administrator.

$site = Get-SPSite http://sharepointsite/sites/MySiteCollection
$RemoveThisGuyAccount = "DOMAIN\BadSiteAdmin"
$AllSiteAdmins = $site.RootWeb.SiteAdministrators
$RemoveThisGuy = $AllSiteAdmins | where {$_.UserLogin -eq $RemoveThisGuyAccount}
$RemoveThisGuy.IsSiteAdmin = $false
$RemoveThisGuy.Update()

SharePoint Duet Enterprise – Importing the BDC Models

I have been following the Microsoft TechNet article instructions to install BDC Models to SharePoint, I executed the command line
C:\Program Files\Duet Enterprise\1.0> DuetConfig.exe /importBDC "DRIVE:\PATH\models.xml" /Addusers "END_USERS" /Username "USERNAME" /Password "PASSWORD" /Email "EMAIL_ADDRESS" /BDCServiceApplication "BDC_SERVICE_APP_NAME"

The command line responded with the error message

Starting import of models from DRIVE:\PATH ...

Attempting to import model: ENTITY
Creating Secure Store application WSDL to store credentials, required to fetch WSDL from SAP Backend for importing the BDC models.
[ERROR] Failed to create target application in Secure Store. Error Message: Failed to create Secure Store Application WSDL. Error: Unable to obtain master key..
Fix the previous error and retry.
Failed to import model: ENTITY

The SharePoint ULS logs recorded an error message in the Claims Authentication category.

SPSecurityContext.WindowsIdentity: Could not retrieve a valid windows identity for NTName='DOMAIN\account', UPN='account@DOMAIN'. UPN is required when Kerberos constrained delegation is used.
No windows identity for DOMAIN\account.

I’ll put the full stack trace at the end of the post.

The SharePoint farm is made up of two web front end servers and two application servers. The Business Data Connectivity service instances and Secure Store service instances were running on the two application servers only. I executed the command on one of the application servers.

The Claims to Windows Token Service was not started on any of the servers. Why should it be? There isn’t any mention of this service in any of the deployment steps via TechNet.

RESOLUTION: I started the service through SharePoint Central Administration on both of the application servers and re-ran the command to import BDC Models. In this scenario, the service identity is “Local Service” and not a domain account. And they all lived happily ever after.

---

DATE TIME w3wp.exe (0x1278) 0x1DF4 SharePoint Foundation Claims Authentication bz7l Medium SPSecurityContext.WindowsIdentity: Could not retrieve a valid windows identity for NTName='DOMAIN\user', UPN='account@DOMAIN'. UPN is required when Kerberos constrained delegation is used. Exception: System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at net.pipe://localhost/s4u/022694f3-9fbd-422b-b4b2-312e25dae2a2 that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. ---> System.IO.PipeException: The pipe endpoint 'net.pipe://localhost/s4u/022694f3-9fbd-422b-b4b2-312e25dae2a2' could not be found on your local machine.      --- End of inner exception stack trace ---    Server stack trace:      at System.ServiceModel.Channels.PipeConnectionInitiator.GetPipeName(Uri uri)     at System.ServiceModel.Channels.NamedPipeConnectionPoolRegistry.NamedPipeConnectionPool.GetPoolKey(EndpointAddress address, Uri via)     at System.ServiceModel.Channels.CommunicationPool`2.TakeConnection(EndpointAddress address, Uri via, TimeSpan timeout, TKey& key)     at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)     at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)     at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)     at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.IdentityModel.WindowsTokenService.S4UClient.IS4UService_dupUpnLogon(String upn, Int32 pid)     at Microsoft.IdentityModel.WindowsTokenService.S4UClient.c__DisplayClass1.b__0(IS4UService_dup channel)     at Microsoft.IdentityModel.WindowsTokenService.S4UClient.CallService(Func`2 contractOperation)     at Microsoft.SharePoint.SPSecurityContext.GetWindowsIdentity().

DATE TIME w3wp.exe (0x1278) 0x1DF4 SharePoint Foundation Claims Authentication g220 Unexpected No windows identity for DOMAIN\account.

Forefront Server Protection for SharePoint 2010 silent install

A silent install of Microsoft Forefront 2010 for SharePoint will cause this message to display.

A program running on this computer is trying to display a message.

Pat Altimore has a blog post describing the technical details of this message at http://blogs.msdn.com/b/patricka/archive/2010/04/27/what-is-interactive-services-detection-and-why-is-it-blinking-at-me.aspx

So Forefront install seems to be running some function in session 0.

When performing an install of Forefront via the GUI we can see that it is performing an STSADM command. The TechNet article explains about the properties that need to be set http://technet.microsoft.com/en-us/library/cc288097(v=office.12).aspx

The MSDN article though is specifically for Office 12 and WSS 3.0, back in the days when STSADM was the way to go. I am unable to find an article related to SharePoint 2010. So I guess the Microsoft Forefront product guys have not been able to update from their bad old STSADM days to new and supported PowerShell cmdlets.

So in the meantime, just tell the warning dialogs to go away or ignore them and your silent install will be fine.

New-SPTrustedRootAuthority New-SPTrustedServiceTokenIssuer The name exceeds the maximum allowed length

I received this error on a PowerShell script for cmdlets New-SPTrustedRootAuthority and New-SPTrustedServiceTokenIssuer

The name exceeds the maximum allowed length.
    + CategoryInfo          : InvalidData:(Microsoft.Share...viceTokenIssuer:SPCmdletNewTrustedServiceTokenIssuer) [New-SPTrustedServiceTokenIssuer], ArgumentException
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletNewTrustedServiceTokenIssuer

This actually has nothing at all to do with lengths of anything, but is caused when the Service Token Issuer certificate is not paired with a Trusted Root Authority.

In a scripted environment build, the commands were similar to below (names changed to protect the innocent).

(On the Providing Farm)
New-SPTrustedRootAuthority $consumingRootCert -Certificate $trustCert
New-SPTrustedServiceTokenIssuer $consumingSTSCert -Certificate $stsCert

The commands use the first parameter as an identifier for SharePoint. The value of the parameter MUST be the same for the two certificates coming from the same environment. For example,

New-SPTrustedRootAuthority "ConsumingFarmCertificates" -Certificate $trustCert
New-SPTrustedServiceTokenIssuer "ConsumingFarmCertificates" -Certificate $stsCert

If anyone comes across this post as a result of meeting this error and can’t interpret my advice, let me know by posting a reply and I’ll see where I can improve the explanation.

SharePoint Enterprise Search – crawling Last Modified By

I was investigating why SharePoint search results were not including the refiner for Last Modified By when I came across this information.

SharePoint Enterprise Search requires certain settings in order to crawl data for Last Modified By and add to the search index.

Firstly you need a managed property – there is a default managed property called ModifiedBy but you can make your own if you wish. From the Enterprise Search Administration page, navigate to the Metadata Properties page.

The Metadata Properties page shows all managed properties for the Enterprise Search service application.

The managed property needs to be mapped to the crawled property “Office:8(Text)”. The crawled property needs to have ticked tickbox for “Include valies for this property in the search index”.

If anyone can tell me how Office:8(Text) happens to map to the “Last Modified By” attribute of an Office document, it would be much appreciated. In fact, a prize to go out to the first who can find TechNet documentation on what all the Office crawled properties are.

Finally don’t forget to run a crawl before seeing this in search results.

The PowerShell script equivalent of the above is as follows:

$searchApp = Get-SPEnterpriseSearchServiceApplication
$manangedProperty = Get-SPEnterpriseSearchMetadataManagedProperty -SearchApplication $searchApp -Identity "ModifiedBy"
$crawledProperty = Get-SPEnterpriseSearchMetadataCrawledProperty -SearchApplication $searchApp -Name "8" -Category "Office" -PropSet "f29f85e0-4ff9-1068-ab91-08002b27b3d9"
$crawledProperty.IsMappedToContents = $true
$crawledProperty.Update()
New-SPEnterpriseSearchMetadataMapping -SearchApplication $searchApp -ManagedProperty $managedProperty -CrawledProperty $crawledProperty