The CRM Grid – A Microsoft CRM Blog

Greg Owens’ rose-tinted MS-CRM mumblings

Corrupted entity – NullReferenceException when publishing or exporting

A corrupted entity
I ran into a problem today with my client which frankly had me stumped. For some reason that I have yet to determine (though possibly related to a broken install of Update Rollup 2), the account entity got corrupted. We discovered this when somebody attempted to publish their changes to the account entity and ran into a “red” error dialog: “An error has occurred…please contact your system administrator”. This error was replicated if we tried to publish, delete any attributes from the entity or export the entity customizations. Not much scope for fixing the situation!

Troubleshooting…
After studying the server trace logs (thanks to CrmDiagTool4 and Michael Hoehne’s Trace Log Viewer for easing this process), the situation wasn’t much clearer. The underlying error was:

Request URL: http://localhost/MyOrgName/AppWebServices/SystemCustomization.asmx
Stack Trace Info: [NullReferenceException: Object reference not set to an instance of an object.]
at Microsoft.Crm.ObjectModel.OrganizationUIService.LabelLoaderAllLanguages.LoadMetadataLabel(Int32 entityType, String attributeName, ExecutionContext context)

I also did a SQL Trace using SQL Profiler on the server, but this didn’t really provide anything new at all.

So, when faced with a problem I can’t resolve myself, I turned to Google for help. I managed to find KB article 947096 which seemed very near to my problem, but sadly none of the resolution steps would help me. I then tried importing a known good copy of the account entity customizations but that failed too. At this point I looked back at the Trace Log and tried to determine what was going on “under the hood” prior to the failure…

It was clear from the logs that CRM was trying to pull in LocalizedLabel information for my entity attributes and form controls, however I couldn’t quite fathom out which attribute was causing the NullReferenceException (it seems that the last statement in the logs – MetadataProcessObject.ExecuteQuery – was successfully completing its SQL command and the following statement which was failing was not logging any parameters and therefore didn’t provide me with a “smoking gun”). In fact there were many calls to both CrmDbConnection.InternalExecuteReader and MetadataProcessObject.ExecuteQuery so clearly there was an interative loop going on. I didn’t know which mechanism within the CRM server black box was selecting the list of attributes to process, so I looked further back and found that this loop was likely being “triggered” by the FormXML property on the OrganisationUI view.

At this point everything clicked into place: the problem, somehow, was with the published version of my form (some of you may have realised this long ago). This form xml was being re-evaluated (for some reason) prior to every publish, export or delete request and then failing some integrity or logical check. Re-importing a known good copy would not help, since it would need to be published to take effect. I only had one option left – to break all the rules and manually amend the database in order to correct the corrupted FormXml property…

So here’s what I did:

Resolution
Took a backup of the current formXml property by running the following SQL statement within SQL Server Management Studio in the “_MSCRM” database (then right click on results tab and click “save as”…):


select organizationui0.ObjectTypeCode organizationui0.FormXml as 'formxml',
from OrganizationUI as organizationui0
where (organizationui0.InProduction = 1 and (organizationui0.ObjectTypeCode = 1))

Next I found my “good” copy of the customizations file. I identified the start and end node that was required in order to match the same format as the existing formXML property (the starting tags were <form><entity…>). Next I copied this xml to a new file and removed all tabs and new lines, to give a single continuous string.

I saved this xml file onto my CRM server then went back to SQL Management Studio and ran the following SQL to import this long string directly into the SQL table:

declare @xmlText varchar(max)
select @xmlText =(select * from openrowset (bulk 'c:\amendedformxml.txt',SINGLE_CLOB)x)
update dbo.OrganizationUI
set FormXml = @xmlText
where (InProduction = 1 and (ObjectTypeCode = 1))

This effectively “forced” my old formXML into a published state and magically fixed my problems. Further, once I clicked “Publish” following these steps, the formXML was consequently re-evaluated again and the hitherto unpublished customizations become published (as expected) and therefore effectively no information had been lost by importing the older xml.

Conclusion
So I still don’t know what caused this issue… I’d love to know if anyone else has had a similar situation. I’m not entirely comfortable with the way I had to fix the problem either, but then sometimes your hand is forced into doing things you don’t want to do. I’ll monitor the situation and let you all know if there are any repercussions. In the meantime, the usual caveats apply – the steps above are definitely unsupported and could definitely render your CRM installation inoperable, so use the steps above at your own risk!

Advertisements

9 Comments»

  David wrote @

Hi,

I had exactly the same problem and I was very glad I found your piost. Unfortunately, your trick with importing old formXML didn’t work for me. Probably, because I could locate a version that would work. I tried with newly imported one and copy from my development server.

What I ended up doing, was deleting all field from unpublished entity form, then doing sql update that copied formxml with inProduction = 0 over the one with inProduction = 1. Than I was able to publish, reimport customizations and publish again.

  Greg Owens wrote @

I can just see the CRM support team writhing in their seats in horror if they ever read our post/comments – but good work David! Thanks for sharing your solution!

  David wrote @

I wouldn’t worry about the CRM team. We are doing this, because of them. 🙂

I still have a problem though… I can repeat the whole circle thing, but I’m still not sure how I can fix it. When I import a custom entity some attributes a relationships are missing. Since they are used on the form, the problem above happens in the first place. You would have any idea, why my attri butes get lost, would you? O:-)

  Greg Owens wrote @

Where are you importing from? Has the source system been published (“Publish ALL” to ensure any related entities/ relationships are published too). If you have relationships to other entities, ensure they are exported and imported. It seems most likely that the attributes missing are from a related entity though – can you confirm such a link?

If that fails, consider importing FROM the target system and into your source system, publish, then try importing from the source and BACK to the target (as you originally tried).

  David wrote @

Hi,

I managed to solve my problem. I exported all customer entities at once. Duh.

  Augusto wrote @

Thanks for your posting! I had a slightly different corrupted database but your solution worked anyway.
Thanks again.

  Grant wrote @

Greg, David,
My monitor was almost the recipient of a frustration head-butt when I stumbled on this post. It saved my life. Swapping for a clean entity formXML didn’t seem to work but overwritting unpublished with published worked (thanks for the supplement David).

Thanks,
Grant

  Marc-Allen Johnson wrote @

Greg, one word: Awesome

After two days of googling, tracelog and sqlprofiler torture, we were also considering this unsupported route. (why does this specific problem always happen just before the deadline?). Your experiences echoed ours exactly and the solution solved the problem within 15 minutes! In our case it was the Contact entity (ObjectTypeCode = 2).

We had to fix dev (backup? what backup?), so we used the production formxml. This worked fine for us – but it might depend on how heavily you customised.

If this blog entry ever becomes a “Wisdom of the ancients” (http://xkcd.com/979/) post, fellow traveller, your search is over.

If on the other hand you feel the need to experience the nightmare yourself (like for instance having to prove to your manager to take the unsupported route), try the solutions in the following thread:

http://social.microsoft.com/Forums/en/crm/thread/f6de6b65-29e7-4238-a042-c808ee65c026

Greg, thanks for helping me sleep at night, knowing that I don’t have to read one more “o, it is easy, simply compare the form fields to your attribute list” piece of advice.

  Greg Owens wrote @

Thanks for the kudos Marc-Allen, it’s great to feel like I provided at least a little help.

(love that xkcd post – been there too many times!)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: