[wpkg-users] Replace xmlhttp download with curl/wget

[Rainer Meier]

Hi Marco,

On 15.07.2011 11:32, Marco Gaiarin wrote:
> + WSH, as stated in this list, have poor performance and limited
>   features when downloading

This is absolutely clear. That's also a reason why I do not really support the 
<download /> tag in particular. I think it's just a legacy feature for very 
basic download functionality. From my point of view most people would be better 
off by using wget and similar tools.


> + WSH, as stated again here, have no hash function, any hash algorithm
>   have to be implemented in pure WSH or relay on an external script

Sure, that's why I don't think using the an md5 attribute in <download /> makes 
much sense. As this would be either ignored in default case (pure WPKG 
deployment without wget/md5sum). Up to now all the features of WPKG are 
available by the default wpkg.js without the strict requirement for external tools.


> So seems reasonable to, optionally, relay on an external script; seems
> reasonable, because we can relay on an external script, to add the
> 'md5' field to<download />  tag, that the internal WSH downloaded
> ignore but can be passed, optionally, to the external script.

As said above some features discussed in this thread would then require 
additional tools (these tools would then not be optional again). So use of any 
of these flags would break the package if the tools it relies on are not available.
Moreover such a hardcoded implementation would add a dependency to SPECIFIC 
download/md5 tools like wget and md5sum. Alternative tools cannot be easily used 
as the interface might be different. For example one might like to use sha1 
checksumming or prefers an md5 sum tool which is faster but prints the output in 
slightly different format etc.


> You are right, we can just use existing<install />  and<update />
> tags to do so, but the<download />  tag just exist, and i think is
> more cleaner to use it.

Yes it might be more "clean" but using commands is much more flexible and allows 
you to use any tool which best fits your environment (and not just one which is 
chosen by WPKG team and enforces you to use this one).

So for me there would be only one compromise we could potentially think about:
Ship WPKG with a download-helper script.
E.g. ./tools/download/wpgk-downloader.cmd
This script accepts a clearly defined list of parameters like:
- Download source URL
- Download target folder
- MD5 sum in pre-defined encoding (optional)
- SH1 sum in pre-defined encoding (optional)

The script could then decide on some internal logic what to download and whether 
the download was successful or not and then return with pre-defined exit status.
E.g.
0  all success, download finished, checksum OK.
1  download failed
2  any checksum failed (md5 or sha1)

However I still think it might be easier to provide such an optional script to 
be used in commands rather than called from download nodes in package.xml. The 
reason is simply that I see that commands would already support any number of 
parameters, using environment variables etc. while using a strict download tag 
structure always limits to certain features. So if later sha2 support or any 
other special solution is required, then it would requiring to adapt wpkg.js 
code. Using commands will work no matter which parameters might be added later.

> [ Indeed, this have nothing to do with my ''server side download'' script,
>    using the global /noDownlaod switch on wpkg.js.
>    Clearly, also my script can be easily extended to use md5 hashes... ]

Sure, and even when using commands to download files it could be globally 
disabled by simply supporting an environment variable NO_DOWNLOAD which makes 
the script to skip its task and exit with code 0 instead (or alternatively still 
verifying the checksums).

br,
Rainer