Eric Nagel

Eric Nagel

CTO, PHP Programmer, Affiliate Marketer & IT Consultant

Building a Datafeed Site – Step 2

I have been, or can be if you click on a link and make a purchase, compensated via a cash payment, gift, or something else of value for writing this post. Regardless, I only recommend products or services I use personally and believe will be good for my readers.

By now, we have selected the niche and chosen a template, added some supporting files and created the database, and are now ready to populate the products table with actual products!

The first step, if you haven’t done so already, is to download the datafeed from the ShareASale website. While many merchants offer FTP access, and we could automate this a bit more, it’s easier when getting started to upload the data manually (yeah, uploading a zip of the datafeed is “manual” to me).

You’ll need to create a “temp” folder, chmod it to 777 so the script can write to it, and maybe change where your unzip program sits on your server (line 17)

The script to load the products table isn’t terribly long, but I’m not going to paste the whole thing in this blog post. So…

Download the file here

Then, upload the file to your website, go to yourdomain.com/admin-sas-datafeed.php and click on the “Choose File” button, then select the zip file from your computer, and click “Upload”.
Uploading data

What this does is unzips the .zip file, reads the .txt file and if there’s a new product, adds it to the database. If the product already exists, it gets updated. Finally, any product that wasn’t just added or updated, gets removed.

Repeat as necessary for each merchant that you’re working with.

This step isn’t very long, or difficult (considering I give you everything you need!) but it’s exciting as now you have a database full of products, ready to display on your website!

Next up… we’ll work on the display of the products.

Datafeed Site Navigation

Comments
  • Panda
    Posted June 29, 2010 3:17 am 0Likes

    Hey Eric, when do you swap in your affiliate ID?

  • Eric Nagel
    Posted June 29, 2010 9:31 am 0Likes

    @Panda: about line 75
    $rsItem[$rsMap[‘Link’]] = str_replace(‘YOURUSERID’, $nSASID, $rsItem[$rsMap[‘Link’]]);

  • Panda
    Posted June 30, 2010 1:15 am 0Likes

    Oh yeah, I forgot about the vars file.
    /me slaps head.

    James was wondering if it was only for SAS and I said yeah. Others want to know if it will work with WP, as well as I. Now that WP3.0 is out, I wonder if it were possible.

    Oh well.

  • trouble
    Posted August 4, 2011 12:51 am 0Likes

    I am using WP, have split my datafeed into 500 line chunks, and use csv import plugin after I have massaged the datafeed a bit.

  • Jon
    Posted December 19, 2011 2:58 pm 0Likes

    Hi, first of all, thanks for sharing your code. Unfortunately I was not able to make it work.I am getting the following error.
    Warning: fopen(26485.txt) [function.fopen]: failed to open stream: No such file or directory in /home/johnny01/public_html/mysite.com/admin-sas-datafeed.php on line 29 ($fp = fopen($cTxtFile, “r”); )

    Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite.com/admin-sas-datafeed.php on line 74 (while (($rsItem = fgetcsv($fp, 100000, “|”)) !== FALSE) { ) please advice.thanks.

    • Eric Nagel
      Posted December 19, 2011 3:05 pm 0Likes

      Just guessing, but the script may not have permission to write the file after unzipping.

  • Jon
    Posted December 19, 2011 4:30 pm 0Likes

    wow thanks for the fast feedback.
    Eric, I noticed the temp file I created with 777 permissions has 4 files out of these files 3 start with the prefix core.(follow by a number) eg. core.13652 and so on their sizes ares 8.03MB each. the other file is an error_log (256MB) also this particular merchant has its datafeed as text file. one more thing in your code line 75 $rsItem[$rsMap[‘Link’]] = str_replace(‘YOURUSERID’, $nSASID, $rsItem[$rsMap[‘Link’]]); I have replace ‘YOURUSERID’ with my ShareASale user ID number.. as suggested by my merchant instructions: You will need to replace every occurrence of YOURUSERID in this file, with your actual ShareASale userID number, which is 111111. Your browser will probably ask you if you want to Open the file, or Save. Choose “Save”, and save the file to your computer…from there you can open it in any program you wish.

    Fields are seperated by the “|” “Pipe” symbol, not by the comma. this may be a silly question but in your code str_replace(‘YOURUSERID’, $nSASID, are not the same ? store in vars.php, maybe I am missing something simple here, hope you can help me to understand and then figure it out. thanks again.

    • Eric Nagel
      Posted December 20, 2011 7:40 am 0Likes

      Hi Jon,

      Line 75 handles that replacement for you, so leave it as-is. You set your ID in vars.php, which you get in Step 1b

      Are you trying a very, very large datafeed?

      Are you putting this all together step-by-step, or are you using the final zip of all of the files?

      • Jon
        Posted December 20, 2011 10:16 am 0Likes

        Hi Eric,I have contacted my hosting provider and they advised me to change
        chdir(‘./temp/’);
        with
        chdir(‘./home/johnny01/public_html/mysite/temp/’);
        this time the .txt (274Kb) file got put to the new “temp” file location
        These are the new errors:

        Warning: chdir() [function.chdir]: No such file or directory (errno 2) in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 16
        chdir(‘./home/johnny01/public_html/mysite/temp/’);

        Warning: unlink(26485.txt) [function.unlink]: No such file or directory in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 23
        unlink($_FILES[‘sas’][‘name’]);

        Warning: fopen(26485.txt) [function.fopen]: failed to open stream: No such file or directory in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 29
        $fp = fopen($cTxtFile, “r”);

        Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 74
        while (($rsItem = fgetcsv($fp, 100000, “|”)) !== FALSE) {

        I am doing this step by step ,my id is set in vars.php, datafeed size is only 274kb . thank you again. Jon

        • Eric Nagel
          Posted December 20, 2011 10:54 am 0Likes

          Remove the . before ./home

          chdir(‘/home/johnny01/public_html/mysite/temp/’);

          (WordPress will probably make those “smart” quotes, so make sure you use a regular, single-quote

          • Jon
            Posted December 20, 2011 11:56 am 0Likes

            Hi Eric, I made the changes but did not work( I forgot to mention, I am not using WordPress) I noticed something on the table products I have ;field #1 ProductID and the last one is field #42 bActiveproduct.
            on your script:
            $rsMap[‘ProductID’] = 0;
            $rsMap[‘Option5’] = 40;
            thanks for your help.

          • Jon
            Posted December 20, 2011 2:34 pm 0Likes

            Eric,this is the transcript report from hostgator.com.

            Hello Johnny,

            Thank you for contacting support! We are happy to help! There is an error_log file in the temp directory, I would appear that it is expecting a resource and you supplied a boolean variable and that the execution time waiting for this was exceeded.

            [root@gator1214 /home/johnny01/public_html/mysite/temp]# tail -f error_log
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Warning: fgetcsv() expects parameter 1 to be resource, boolean given in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 78
            [20-Dec-2011 10:48:53] PHP Fatal error: Maximum execution time of 30 seconds exceeded in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 93

            I hope you can interpret this error_logs better than I do. thanks.

        • Eric Nagel
          Posted December 20, 2011 2:42 pm 0Likes

          Sorry it’s not working for you. I can only debug so much without having access. Either try downloading the final solution, or contact me about doing the job for you

          • Jon
            Posted December 21, 2011 12:39 pm 0Likes

            Hi Eric, I finally get it to “work”, We know the file got moved to the “temp” folder but we got errors: this is one of them.
            Warning: fopen(26485.txt) [function.fopen]: failed to open stream: No such file or directory in /home/johnny01/public_html/mysite/admin-sas-datafeed.php on line 29
            $fp = fopen($cTxtFile, “r”); ( I just replaced this with
            $fp = fopen(“/home/johnny01/public_html/mysite/temp/26485.txt”, “r”); and got echo : The products have been updated. Now I can see the table ‘products’ populated,Next,I am going to start step 3. thanks for all your help! 😛

  • Kenny
    Posted September 25, 2013 4:51 pm 0Likes

    Mr. Nagel,

    Nice FREE tutorial on datafeeds…

    I was able to do exactly what you instructed, even including the css.

    Only one minor item that is easily fixed.
    1. Undefined variable: rsMap in admin-sas-datafeed.php on line 26
    // Set up a map to easily reference the data
    $rsMap[”] = count($rsMap);
    2. The YOURUSERID is not being replaced by vars.php. Which is related to error above.

    Conclusion, very nice script, their is so much you can do with this. Being able to utilize the custom fields and data filters will come in handy!

    Having issue with this tutorial? Try going through this at least three time, from step #1. Unzip, Create new database, etc… It’s pretty flawless if not rushed through!

    This is out of the box free stuff and I’m thankful!
    Nicely Done Eric!
    Regards,
    Kenny

Leave A Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.