Category: E-commerce

PayPal IPN, PDT & Analytics tracking, getting there.

As I’ve previously written about, I’ve been having some trouble with PayPal PDT ( Payment Data Transfer ) and Google Analytics e-commerce tracking.

If you’ve added Analytics to your PayPal thank-you page, and used PDT to get the data to be sent to Analytics and got that working nicely then you will have discovered that not every sale gets tracked, because not every shopper can be guaranteed to land on your thank-you page.

Now I’ve been doing some testing on an idea to get around this using IPN. The basic premise is this.

  1. Google Analytics works through a request for __utm.gif from the Analytics server, to which is attached all the tracking information.
  2. On the page just before leaving for PayPal set the analytics script to local only.
  3. Use url rewriting to hide a script behind your local __utm.gif.
  4. Record all the details for the request for the local __utm.gif in a database, referenced to the session.
  5. Send the session ID information through PayPal’s custom variable.
  6. User finishes sale.
  7. IPN script picks up sale, checks session ID and looks up the stored request.
  8. IPN script rebuilds request and forwards to Google’s __utm.gif.

Now to test this before getting involved in databases and IPNs I just made two pages. One with Analytics set to remote, the next page with analytics set to local. Then I made a local __utm.gif just forward the request for the remote __utm.gif through a server side HTTP request.

Wait a day.

No joy, nothing showing up for the e-commerce tracking, or the visit to the second page, just the first page visits where the script was set to remote.

Then I fiddle for weeks trying to improve the request. Adding all the cookies etc. etc.

No joy.

So I took a look at the urchin.js file to determine whether or not the local request was being built any differently to the remote one. And it was. If you have a look through urchin.js searching for the local/remote variable (forget the name of it right now) then you’ll see that there are extra things appended to the end of the querystring for the remote call to _utm.gif. The extra stuff is the content of all the analytics cookies.

So I fiddle urchin.js and made a local copy that built the same request for remote and local, including all the cookie data. I had a couple of people hit the two pages and a day later…

Success, there is e-commerce transaction data, items, totals, everything in Analytics now.

I used different sources and the like for a couple of test and those have also come through. Even more surprising though is that where I have had other people test the two pages for me, their locations have been tracked and assigned to the e-commerce transactions correctly. Which I didn’t expect as it’s always the web server making that request, and it’s not moving around!

So that’s something to work on. I’ll keep you updated on the next stage of implementation.

PayPal E-commerce Tracking with Google Analytics

Updated post: http://www.teknohippy.net/2008/03/04/paypal-ipn-pdt-analytics-tracking-getting-there/

Trying to track E-commerce transactions with Google Analytics (GA) and PayPal can be problematic.

PayPal has two ways of returning data to your server. Payment Data Transfer (PDT) and Instant Payment Notification (IPN).

PDT can be used to setup GA tracking by including the relevant GA scripts on a receipt page, processing the PDT data server-side and populating those scripts. Instruct PayPal that this page is your “return” page and when visitors return to it the e-commerce and goal data will be tracked.

The big problem with this is that PayPal users do not have to return to the receipt page, even if you’ve turned on the auto return feature. The pause before redirecting is very long and many people just abandon on the PayPal pages.

IPN on the other hand will always get triggered. The problem is it calls a script on your server from PayPal’s server-side scripts. This means your IPN script never runs in the users browser and so the GA JavaScript is never executed and no tracking will happen.

Now at it’s heart all the GA script really does is assemble a call to an image on the GA servers, __utm.gif to be more specific. Appended to this request is a lengthy querystring with all the information for Analytics to log.

It should therefore be possible to capture users info or the actual __utm.gif querystring on the page just before leaving for PayPal, temporarily store this in a database against the users SessionID, and pass the SessionID to PayPal as a custom pass-through variable.

The easiest way to collect the querystring from the pre-PayPal page is to set “_userv = 2;” in the tracking code. This means that __utm.gif will be requested from both the Analytics server and your server. Trap the call to _utm.gif with some server-side script and you’ve got the whole querystring very easily, as well as the corresponding SessionID.

The user is now off to PayPal to complete their transaction.

Then in your IPN script, lookup the SessionID and recover the __utm.gif querystring, change the utmp path variable for the path of the reciept page and then use your server-side script to make an http request to __utm.gif.

This will then at least track and goal reporting will work. The IP address will be wrong but the rest of the data will be correct.

More complex is building the querystring for the  __utmSetTrans() call. I’ve had a look into what’s sent and a separate call is made to __utm.gif for the transaction and each item within it.

I suppose it would be possible to set _userv to 0 and only have a local __utm.gif request on the checkout page. Then the Ecommerce tracking data could also be included. All the script hiding behind the local __utm.gif needs to do is record all requests when visitors hit the checkout page. It can pass on the actualy request for the tracking (but not the transaction), when it happens. Then in the IPN script the tracking request path can be rewritten and replayed, followed by the transaction requests. Sounds like it should work.

I guess I should go make it happen and test it!