Last time I wrote about doing imports of large data sets when using AFIncrementalStore to handle server synchronization I thought I had made an end-run around it, finding a technique which was compatible but faster. I was wrong.
Before I begin, this is not a knock on AFIncrementalStore which does its job well overall. It was my first use of this library, and I have no trouble recommending it for certain uses cases.
After my last post, when I started to test more thoroughly and use the imported data, I saw strange things. Mostly that meant my objects relationships were not what I expected. I’m fairly sure I had seen these relationships created as they should be, but in the fog of the battle which was my work on this feature, I can’t say for certain. My bad.
A quick refresher on what I was doing in an effort to speed up a large first-time sync with the server. I requested the import data in batches and created NSManagedObjects from that JSON. In a second pass over the objects I set up the relationships. All that was left was to call save: on the managed object context. Easy and fast. Then I started using the data and saw, or rather didn’t see, the relationships I was expecting.
So what went wrong?
When NSManagedObjects are created Core Data gives them a temporary ID. Your object’s model might include a unique ID, but these are different. AFIncrementalStore cares about them (and it cares about your model’s unique ID). When the objects are saved they will be given a non-temporary ID.
While saving, AFIncrementalStore has the responsibility of transferring the objects to its private backing store. For each object saved, it updates its backing object with the new attributes and relationships. Except that with relationships, if the object being saved has a temporary ID it just ignores the relationship. Silently.
Core Data provides a method on NSManagedObjectContext to assign an object a permanent ID. This would make everything work nicely. Again, AFIncrementalStore is given the opportunity to manage permanent IDs. This part is more complicated, but it boils down to AFIncrementalStore doesn’t have the information it wants in order to assign the permanent ID, and it leaves the temporary ID as is. Silently, as is its way.
The extra information AFIncrementalStore wants is the unique ID as the server sees it. Since it didn’t make the server request, it doesn’t have that information. If it did, it would store that value in its special property it adds to your model, and stash some private information which doesn’t persist. There is no public method to tell it which unique property to use or what its value is.
I tried a few more things but ultimately I have settled on using batched NSFetchRequests which let AFIncrementalStore do what it does.
This was a tough battle, and hopefully you’ll be spared the trouble. In the end I wonder if Core Data is really the right choice when dealing with a few 50,000+ row tables of data. Some think not. I am sure that doing straight calls to a database or some more direct framework would have given the performance I was seeking. But I had to weigh that against the benefit I was getting from Core Data and AFIncrementalStore in other areas, and I chose to keep using it.
