- This topic has 18 replies, 6 voices, and was last updated 10 years, 10 months ago by
.
-
Posts
-
Hi, I am using eStore to sell mp3s, pdfs and mp4s. Most of these are quite large and are held within zip files.
All the product download files are held on an external server – Rackspace Cloud Server and the containers which hold the files are made available to access via the public internet by assigning a CDN URL to them. These could look like this: [http://538d9c65b3d645be1d3f-b2e7144e4f0099f41104a2d93727f758.r32.cf1.rackcdn.com/videos.zip]
(please note this is not an actual URL to the product file as I do not wish to post this here).
I had set eStore to encrypt the files using the random code and also to use shortened URLs.
All files can be accessed and downloaded properly via the Rackspace server or via direct browser link, i.e. cutting and pasting the CDN URLs directly into the browser, so I know that the actual links work and that the files are downloaded in full.
Somewhere in the encryption process it seems to go wrong, where the files are supposedly being downloaded, but instead of downloading the full 620 MB, it only downloads 280 KB when using the eStore generated link.
This therefore doesn’t appear to be a permissions or a download limit problem as the direct links to the Rackspace files are working properly.
I have tried it with switching the shortened links off, but this has not made a difference.
Your documentation says that storing the product files on an external server is fine as long as the files can be accessed in a browser. Well, they can, so I am at a loss to pinpoint where things are going wrong.
It is not just zip files that are not being downloaded fully by the way, it also happens to mp4s and mp3s.
These are the Download Manager log files of 30th June, 2013 (please note I have replaced all paths with the bogus one mentioned above:
[06/30/2013 12:58 AM] – Warning : Didn’t find a direct match for the domain URL in the src file.
[06/30/2013 12:58 AM] – Warning : Trying a secondary URL conversion method.
Would you be able to look into this? I am willing to give you the actual links and login information to my WordPress installation, but in a private message if that’s OK.
Thanks,
Louise
When curl’ing to the file I get the following:
$ curl -v [redacted]
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: salesclosingpower.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 30 Jun 2013 01:36:33 GMT
< Pragma: public
< Server: Apache
< Expires: 0
< Connection: Keep-Alive
< Set-Cookie: PHPSESSID=kfv0n7l7rou4qju2dv59dpejk2; path=/
< Host-Header: 192fc2e7e50945beb8231a492d6a8024
< Content-Type: application/zip
< Cache-Control: public
< Transfer-Encoding: chunked
< Content-Description: File Transfer
< Content-Disposition: attachment; filename=”Sales Closing Power Seminar Video Sessions 7-12.zip”
< Content-Transfer-Encoding: binary
Could this be because the file name has spaces in it?
Louise
The way encrypted download links work is by using the server as a middle man between the source file and the user. Normally, the source file is on the server, and it merely needs to read the file and push it out to the user.
By externally hosting the files, you’ve effectively doubled the bandwidth usage of your server; once to download the file from the external source to the server, and once to push the file back out to the user.
Additionally, because the server is now burdened with having to copy/push the file in/out twice between the external source and the user; processing time is increased, which causes HTTP session timeouts to occur. The usual symptom of an HTTP session timeout is when the debug log shows a successful download with no server side errors, but the user gets a truncated (or even a 0k) file.
I also noticed, from the Rackspace CDN promotional material, that they use HTTP compression. This may or may not be factoring into the problem.
If you must externally host files, please use the eStore Amazon S3 integration. Instead of copying/pushing files; Amazon S3 integration works by handing off all of the downloading to the Amazon S3 servers, which then push the files directly to the user.
The cost of Amazon S3 is free for the first year. Users are reporting average monthly costs of 7 to 15 cents a month.
Amazon S3 is reliable; it uses the same network of servers that Netflix uses.
BTW, trying to CURL an encrypted link won’t work, because the encrypted link is not really a file, but a PHP script with a query string; which kicks off the previously described actions.
Thank you, that makes a whole lot of sense. I’ll look into the Amazon S3 option. You may need to update your eStore documentation to explain the doubling of bandwidth issue. If I had known this, I probably would never have started out that way.
But thanks for the heads up – will make some changes!
Louise
This original article eludes to the bandwidth doubling effect.
http://www.tipsandtricks-hq.com/a-bucket-full-of-objects-amazon-s3-3052
I just spent a WEEK configuring my many MP3 and PDF files in HostGator, testing the links, then uploading them to the products in eStore. I also asked a question in this forum and WZP Moderator answered it, but did not tell me I should use Amazon as the external host of file.
I just tested one of the encrypted links by sending myself a test email. The PDFs links worked, the MP3s did not- couldn’t download it.
I searched the forum for a fix and NOW I see this above:
>>>”If you must externally host files, please use the eStore Amazon S3 integration. Instead of copying/pushing files; Amazon S3 integration works by handing off all of the downloading to the Amazon S3 servers, which then push the files directly to the user.<<<
WHY don’t you tell customers this clearly in the eStore so they don’t waste so much time setting up their files elsewhere?!
“If you must…” If I must? I read in this forum that you should not upload your MP3 and PDF files that you sell into WP because people could easily find them if they know what to do. Is this true or not? Would you store the files you sell in WP? I am not tech-savvy so I have no idea if this is necessary.
I looked briefly at the Amazon S3 option and it looked like it’s complicated and a hassle for people who are not tech-savvy. Is it?
Very frustrated at this point….
I know you won’t like the explanation, but it “depends” on your situation.
A. If you use a “top of the line (expensive)” hosting service, you may never run into any problems; and can host everything on your server.
B. If you use a “mid range (moderately priced)” hosting service, you can self-host all but the largest files.
C. If you use an inexpensive or “free” shared hosting service, you will more than likely have problems self-hosting all but the smallest files.
D. Hosting files on an external server will double your bandwidth usage.
E. Hosting files on Amazon S3, using the native Amazon S3 integration feature of eStore, will result in download performance comparable to if you were using a top of the line server.
F. Amazon S3 integration does not work with PDF files that must be processed by the PDF Stamper plugin.
G. In order of performance expectations, based upon your server type, from highest to lowest: Dedicated, Grid, Shared, Virtual
The advice given in the thread you previously referenced will protect any self hosted content that resides on your server.
Setting up Amazon S3 is no more complicated than using self hosting.
@whitelight308, There is nothing wrong with uploading files to your WP. That is actually a good starting point. There is no ONE solution that fits all so we have a range of options. You use the option that suits you best.
Check the following post where we explain this more:
https://support.tipsandtricks-hq.com/forums/topic/download-directory-protection
BTW, the above post is liked from where you configure the digital downloads in a product.
Start somewhere easy that works for you then slowly get into applying more security and protection (there are lots of different options so it will take you some time to learn the pros and cons of each options).
WZP: Thanks for the info.
>>>>>A. If you use a “top of the line (expensive)” hosting service, you may never run into any problems; and can host everything on your server.
B. If you use a “mid range (moderately priced)” hosting service, you can self-host all but the largest files.
C. If you use an inexpensive or “free” shared hosting service, you will more than likely have problems self-hosting all but the smallest files.
What is Hostgator considered? I pay for shared.
>>>>>F. Amazon S3 integration does not work with PDF files that must be processed by the PDF Stamper plugin.
Are there any third-party plugins that would interfere with Amazon S3?
Admin: Thanks for the info. If there’s nothing wrong with uploading files to the WP media library, why does anyone put them on Amazon S3?
Can you please tell me if it’s easy for people to search for and find files stored in the WP media if they know what to do? AND IF SO, WHAT CAN I DO TO PROTECT MY MP3 AND PDF FILES IN THE WP MEDIA LIBRARY?
Thanks for the link, but I read that weeks ago and it says nothing about how eStore encrypted links won’t work for MP3s hosted on a shared server. Please consider warning people so they don’t waste a lot of time like I did. Many people, like me, are not as tech-aware as you guys, so they don’t know these things.
Using a shared server is “okay” for hosting of small files. It is called a “shared server” because you are sharing one physical machine with several other web sites. Because of this, there are many limitations, both computational and by design, that will affect your ability to serve large files for download. If your files are more than a few hundred MB in size, consider using Amazon S3; if you are still in love with your current “shared” hosting plan. People use Amazon S3 integration with eStore primarily because they are using underpowered hosting plans, and it is much less expensive to use Amazon S3 with their current hosting plans than upgrading to a “top of the line” hosting plan.
Encrypted links will work for any kind of file, especially MP3; provided your hosting plan is up to the task.
The advice I gave you in the “digital download url” thread will provide adequate protection for your WP media library files.
https://support.tipsandtricks-hq.com/forums/topic/digital-download-url
However, nothing is 100% hacker proof. There is simply no such thing.
The only known conflict with Amazon S3 integration and plugins, is that you can not use the PDF Stamper on a file that is hosted on Amazon S3. And that is purely a consequence of how Amazon S3 storage works.
Owning a web site is like owning a car. All cars require maintenance and care, but not all owners are auto mechanics. Some people can disassemble and clean their own transmissions, others have ask the dealer to change the oil. When gas stations went to self service, many people were simply unable to deal with the situation.
We have many kinds of customers. The forums are here for those who need assistance. But we can not predict the knowledge level of every person. If you need help, please ask and we will assist you to the best of our ability.
Hi, I am having the same issue as wizard247, but with both files stored on the server and externally: you click on the download link, it downloads about 90ks of the .zip file (out of the 170MBs) and then stops. I have tried with both files hosted on the same server as well as on a different server and still having the same issue. I have asked the hosting provider to look into it already and they say that the issue seems to be with the plugin. Any assistance would be greatly appreciated.
Ok, so after a little more playing around, if I uncheck the ‘Downloadable’ checkbox which encrypts the link, the download works, but now it shows the complete URL for download. Would really appreciate a quick answer to this one!
Did you follow the previously given advice, about enabling the debug logs, reseting them, and then checking the download_manager_debug.log file to determine if any errors occurred?
If no apparent errors appear in the log file, have you tried Download Method 7 (requires cURL library installed on server)?
Have you tried any other Download Methods?
For smaller files, does it work?
What kind of hosting service are you using?
At the end of the day, the cause is usually due to an anderpowered hosting package. And the recommended solution is to use the Amazon S3 integration feature.
Have you checked the following documentation?
http://www.tipsandtricks-hq.com/ecommerce/selling-large-files-with-wp-estore-796
I am having many of these same problems.
I have tried all 8 Download methods.
I am using Amazon S3.
When I generate a link in the Admin Functions, the link is generated ok, and the file downloads with no problem.
But when I create a “Download” button using the Wp estore shortlinks, the only thing that happens it that the page reloads.
I have tried unchecking the “Downloadable” option, as referenced in a comment above, but no change.
My files are not large (around 500kb) so that should not be an issue.
I am using a VPS hosting package, so it is not underpowered.
I did clear the debugging log files, and the only one that has anything is the estore_post_payment_debug.log and it says this:
[12/30/2013 7:36 PM] – Advisory : Log file reset.
[12/30/2013 7:37 PM] – SUCCESS :Generating download link for single file.
[12/30/2013 7:37 PM] – SUCCESS :Generating download link for single file.
[12/30/2013 7:47 PM] – SUCCESS :Generating download link for single file.
I am out of ideas…
When you are hosting files on Amazon S3, you must follow the Amazon S3 integration instructions:
Amazon Web Services Simple Storage Service (AWS S3) Integration
When you are using Amazon S3 integration, all download methods are ignored, because eStore hands off the downloading process to Amazon.
As a minimum, whenever a download button or link is activated, there should be at least 4 entries in the download_manager_debug.log file:
1. That a download link request is being processed.
2. A validation of the download link.
3. A determination of what kind of link:
3a. If it is a file hosted on your server.
3b. If it is an APRTP request.
3c. If it is a (AS3TP) file hosted on Amazon S3.
4. Appropriate processing action(s), determined by previous result.
If debug logs are enabled and the log files have been reset; if after a download attempt, the download_manager_debug.log file is empty, check the file permissions for download_manager_debug.log and ensure they are at least 0664. At the barest minimum, there should be a log file reset message in there.
- You must be logged in to reply to this topic.