PayPal IPN, PDT & Analytics tracking, getting there.

As I’ve previously written about, I’ve been having some trouble with PayPal PDT ( Payment Data Transfer ) and Google Analytics e-commerce tracking.

If you’ve added Analytics to your PayPal thank-you page, and used PDT to get the data to be sent to Analytics and got that working nicely then you will have discovered that not every sale gets tracked, because not every shopper can be guaranteed to land on your thank-you page.

Now I’ve been doing some testing on an idea to get around this using IPN. The basic premise is this.

  1. Google Analytics works through a request for __utm.gif from the Analytics server, to which is attached all the tracking information.
  2. On the page just before leaving for PayPal set the analytics script to local only.
  3. Use url rewriting to hide a script behind your local __utm.gif.
  4. Record all the details for the request for the local __utm.gif in a database, referenced to the session.
  5. Send the session ID information through PayPal’s custom variable.
  6. User finishes sale.
  7. IPN script picks up sale, checks session ID and looks up the stored request.
  8. IPN script rebuilds request and forwards to Google’s __utm.gif.

Now to test this before getting involved in databases and IPNs I just made two pages. One with Analytics set to remote, the next page with analytics set to local. Then I made a local __utm.gif just forward the request for the remote __utm.gif through a server side HTTP request.

Wait a day.

No joy, nothing showing up for the e-commerce tracking, or the visit to the second page, just the first page visits where the script was set to remote.

Then I fiddle for weeks trying to improve the request. Adding all the cookies etc. etc.

No joy.

So I took a look at the urchin.js file to determine whether or not the local request was being built any differently to the remote one. And it was. If you have a look through urchin.js searching for the local/remote variable (forget the name of it right now) then you’ll see that there are extra things appended to the end of the querystring for the remote call to _utm.gif. The extra stuff is the content of all the analytics cookies.

So I fiddle urchin.js and made a local copy that built the same request for remote and local, including all the cookie data. I had a couple of people hit the two pages and a day later…

Success, there is e-commerce transaction data, items, totals, everything in Analytics now.

I used different sources and the like for a couple of test and those have also come through. Even more surprising though is that where I have had other people test the two pages for me, their locations have been tracked and assigned to the e-commerce transactions correctly. Which I didn’t expect as it’s always the web server making that request, and it’s not moving around!

So that’s something to work on. I’ll keep you updated on the next stage of implementation.

20 thoughts on “PayPal IPN, PDT & Analytics tracking, getting there.”

  1. I’m preparing to ship a new product at the moment, so worry about all this is on the back burner for a couple of months now.

    If anyone needs it done and sorted then, to be frank, make me an offer. *grins*

  2. Mate,

    Don’t beg ;)

    If you can get this to work & want a test case to use as more proof, I have just the client for you.
    I’ll pay you to do the work & you can use it to get more.. sound ok?

    send me a note when you get a chance.
    cheers,
    mike

  3. Apologies to anyone waiting on this. It’s still sidelined, may have some time later this week though so I’ve started to think about it again.

  4. I’m definitely very interested in this. Please let me know if you finish developing this and I may be able to convince some clients to send some money your way. Conversely, if there is a specific number that would convince you to expedite completion let me know. ;)

  5. Certainly there is a gathering weight of pressure for me to get on with it. I can’t promise anything at the moment though, got a product to ship.

  6. Pingback: JIRA: Toonstorm
  7. You would figure that after all these years the folks at Google and PayPal would have gotten together to figure out a solution for this. I just launched a site that sells items via PayPal, but it would be ideal if I could track goals (particularly conversions from affiliate programs) via Google Analytics.

  8. One of the easiest, but not cheap, solutions is to sign up for the pro payments PayPal whotsit.

    I’ve still got to get around to fixing this damn thing up and seeing if it really works in a live situation, as the test indicates it might do.

  9. Have you made any more progress on this? I’m trying to use the method of generating the strings myself like you say here and skipping all the javascript stuff. Here’s what I end up with in my page…

    Still nothing showing up in my Analytics Ecommerce, though..??

  10. I just wanted to say that thanks to your post and the steps you laid out I managed to get this working on my site. I’ve got it working with the e-commerce tracking that’s built into Analytics so I’m also getting sales data passed over to GA in the IPN call. It’s going to make a massive difference as I can now look at my stats and actually see conversions … hallelujah! If anyone else wants to implement this, everything you need is in the post above (just read carefully, several times). As Iain says, you do need to use a local, modified version of ga.js. Setting the local parameter isn’t enough – it won’t include the cookie data in the call. Good luck if you go for it and thanks Iain for taking the time to tell us how to do it.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>