slow sync and transparent behaviour
Two users recently tweeted that the addon creates duplicates.
If the addon really created duplicates it’d be a serious bug. Neither of these folk filed a bug report so it’s impossible to know for sure.
But while bugs are always possible, it’s far more likely that these reports reflect a misalignment between expectations and the reality of slow sync.
Even though two contacts might appear to be the same, even a tiny difference (such as a space in a phone number) means that slow sync treats them as entirely different contacts, and one of each appears in the merged set.
The next release of the addon aims to improve transparency during slow sync by explaining:
- what is happening
- why it is happening
- as it is happening
Transparency promotes trust.
Thanks to @zigurd and @ceonyc for the tweets that prompted these reflections.
Happy syncing!
If you liked this Blog, share the love :
Follow on twitter
RSS Feed
April 28th, 2010 at 2:42 pm
The tweets are clearly not useful, but I have had an instance of duplicates during testing that would be helpful to understand.
I had a clean Tb addressbook with 467 entries, created a csv and imported into an empty G contacts list. The reason was to get in my addresses and custom fields into . I backed up both the Tb addressbook mab and the G contacts to a csv.
I installed Zindus, synced, it found no duplicates - perfect. Tested it - again perfect.
Then after a few fast syncs I noticed that while the Tb addressbook had the web pages in the Work field, G had them in the Home Page field. Zindus had noticed this and updated the G contacts so that on G the same webpage now appeared in both the Home Page field and the Work field - again not a problem. No change to Tb as Tb has no “Home Page” field, only “Work” and “Home”. (Not sure on which sync this happened as there is no feedback. Likely the first fast sync?)
I then uninstalled Zindus, deleted the Zindus folder and tested something else, then uninstalled it. I then restored the Tb and G backups and re-installed Zindus.
This time (a) Zindus remembered my G account info and preferences (why?) so I clicked reset to be sure, and when I did my first sync it found 95 duplicates: They were all the entries that had on G the web page in the Home Page field and in Tb in the Work field. This was not too difficult to solve: just amend the backup G csv to change the web pages from Home Page to Work, re-restore this csv, re-restore the Tb backup, reset Zindus, re-run sync, and it found no duplicates.
But the questions are:
. why was the behaviour different?
. how do I get Zindus to really really uninstall and have no memory of previously being installed?
thanks
April 29th, 2010 at 9:39 am
@mfc, I can see you’ve done a bunch of testing! Comments follow.
1. I’m not able to reproduce what you describe re: home url. When I create a contact in TB with http://home.com in the “home web page” field, then when I sync and open gmail, that contact has “http://home.com under “Website” with a “home” label. It sounds like you are seeing something different?
2. I’m guessing that the “duplicates” that got generated after you installed then uninstalled the other addon is because that other addon modified those 95 contacts in some way (eg the postal address field?). So the second time you installed zindus, it did a slow sync and couldn’t match them (because they really were different).
3. It’s playing with fire to (a) have contacts in TB, (b) export them to Google then (c) expect slow sync to match them perfectly. It sounds like it has worked for you and that’s great, but it assumes that Thunderbird’s export process and Google’s import process doesn’t transform the contacts in any way whatsoever and that’s a big assumption! As an alternative, see how to propagate contacts.
4. the reason the addon retained your account details across uninstall/reinstall is because they’re stored in preferences. To see your preferences, open Tools/Options/Advanced/Config Editor then type “extensions.zindus”. You’re probably right that an uninstall of the addon should wipe those prefs too - will file a bug for this.
Thanks for the great feedback!
April 29th, 2010 at 4:16 pm
Leni, thanks. Messing up your contacts can really be disastrous, so testing to make sure Zindus was robust (it is) and that I understood it (see btw at bottom) was important.
1. Yes, when I create a url in the Work / Web Page, csv export / import brings them into G’s Home Page field (G has Home, Work, Home Page, FTP, Blog,…), rather than the Work field. The workaround, once I understood the problem, was to edit the csv before importing.
2. No, as I said I restored the Tb and G backups so that I was in each instance working with clean addressbooks on both sides - many times. The only non-clean item was that in Preferences (4. below) which I could also have restored but did not think it would be an issue.
From what you say the 2nd behaviour (ie 95 duplicates) is the correct behaviour on a slow sync. So I am not sure why it did not happen the first time - the imported csv was certainly the same. It seems like the behaviour that would be expected on a fast sync?? In a long sequence of testing I am bound to have done something I did not properly record.
3. Understand - but that is currently the only way to get across addresses etc. Propagating does not do it. And as I mentioned, addresses, at least for me, are fundamental.
4. thanks, agree.
My guess is that the problem some people had relates to some minor difference in a minor field (eg web page) causing slow-sync to create duplicates. The transparency you have added - saying exactly how many will be added and where - makes it easy to cancel, understand, fix then retry.
btw, A more detailed explanation of how a fast sync works would be helpful.
April 29th, 2010 at 4:23 pm
re 1. I should add I am using Tb 2.0.0.24, which in a csv puts the Work url in Web Page 1 and the Home url in Web Page 2. G them imports them both as Home Page.
April 29th, 2010 at 4:37 pm
@mfc, what sort of things need more explanation re: fast sync?
April 29th, 2010 at 5:47 pm
Leni
eg if a field (say the Work web address) is different between Tb and G, how does it know which is new and which is old (to be overwritten)? Does it keep a snapshot of the Tb addressbook? (seems likely as this is probably one of the functions of slow sync, and a reset might just delete this snapshot) Is there an indicator from G that a field has changed since some event?
April 29th, 2010 at 6:09 pm
@mfc, the addon only knows that a contact has changed, it doesn’t know which field(s) in a contact changed.
This text (added to fast sync) explains what happens next:
When a contact on one side changes, fast sync modifies the contact on the other side by overwriting all the fields that are common to both (and leaving the fields that aren’t common untouched).
April 29th, 2010 at 6:22 pm
Leni
But when you say “When a contact on one side changes, fast sync modifies the contact on the other side by overwriting all the fields that are common to both” they key question is what method does Zindus use to know which side has changed?
A simple comparison would just tell it both sides are different, and it seems like it is not keeping a full copy of each Tb record, so eg does it keep (say) a hash of each record or some other method? Apologies I am not a programmer so can’t check the source myself to understand.
April 29th, 2010 at 7:51 pm
Change detection varies according to the data source:
- tb: calculate a checksum on each sync (as you guessed). Change in checksum implies change in contact.
- Google: query “all contacts newer than XXX” where XXX is the time of the previous request
- Zimbra: query “all contacts newer than XXX” where XXX is a monotically increasing SyncToken
April 29th, 2010 at 8:02 pm
Leni
thanks very much, that is helpful. Better understanding leads to fewer problems
(in life as well as software
May 1st, 2010 at 12:12 pm
@mfc, just FYI, release 0.8.16 of the addon will remove extensions.zindus preferences on uninstall. Thanks for pointing that out!
August 15th, 2010 at 12:14 am
Leni
Just to let you know, have been using zindus intensively now for >3 mths. Absolutely no problems.
Despite G’s recent adding of surname/firstname fields (a few days ago)
which could have created chaos.
thanks
mc
April 30th, 2011 at 1:56 am
There is obviously a bundle to realize about this. I feel you made certain good points in features also.