[wpkg-users] [Bug 261] What about wpkg.js compression?

[bugzilla-daemon at bugzilla.wpkg.org]

http://bugzilla.wpkg.org/show_bug.cgi?id=261

Rainer Meier <r.meier at wpkg.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |r.meier at wpkg.org

--- Comment #1 from Rainer Meier <r.meier at wpkg.org>  ---
Hi,

At the beginning when I've joined WPKG development I've been thinking about
minification (not compression) as well. But there are several drawbacks using
such automatic minification tools.

Maybe the most important one is that nowadays most networks run at gigabit
speed and in worst case at 100Mbps speed. So in a 100Mbps environment it allows
us to transfer WPKG core component about 40 times per second (~25ms transfer
time).

Moreover the purpose of WPKG is to perform complex operations (synchronizing
software). The processing done on client side takes far more processing time.
Even worse if there is something to be installed most applications are by far
more "heavy" than WPKG itself. Most applications are in the 5-100MB range,
requiring transfer and installation.

So I think in today networks (even if on VPN connection with typical DSL speed
of 5Mbps WPKG transfers in less than 1 second). VPN often also uses
compression. Since wpkg.js is pure text it typically transfers even much
faster. Applying standard ZIP compression it transfers in ~80kB.

If anybody is worried on the server load this imposes, then let me tell you
that disk I/O on server side is for sure not a problem. If wpkg.js is requested
many times then any file server I know will have the file in cache in order not
to load it from disk each time.

One more aspect is the size of the WPKG database (XML files) on server-side. In
some environments WPKG will have to load hundreds of small files (or one large
packages.xml) from network. Here it might make a bit more sense to combine
packages to single XML files as SMB2 is pretty slow (especially over small VPN
links) in accessing many small files. It might even be possible to use a HTTP
service instead to present a minified package database to the client requesting
it which contains only the packages used by this specific client.

Unfortunately all these optimizations have an impact on a very important aspect
of the deployment: Maintainability.
In case of error it's much harder to debug a minified wpkg.js. If the package
database is optimized it's getting even worse.

Now we also come to another aspect of JavaScript minification. I've been unable
to find a minification tool which states to be completely WSH-compliant.
Remember that wpkg.js is using JavaScript language but WSH is not 100%
ECMA-Script-compatible. So most minifiers I've tried did not work properly when
applied to wpkg.js. So if we would officially deliver a minified version of
wpkg.js we would have to fully support it and test every single aspect of
wpkg.js with the "normal" and the minified version which is simply too much
effort to be justified by a size reduction of 200kB.

I am not sure about YUICompressor, but I think I've tried this one as well and
it failed to compress the code responsible for environment variable management
(it was broken after minification). Maybe it works with the latest version. But
I am not planning to support it.



The bottom line is that size saved by minifcation would not justify potential
trouble in WPKG use-case. Remember that we are not talking about a JavaScript
tool which is deployed by hundreds or thousands of page requests per second.
Compared to the use-case of WPKG transfer volume and time just does not matter
in most cases.
If you still think it does in your case. Then feel free to use any working
minifier out there and apply it to the official WPKG distribution. Just
remember that in case you run into problems try running the non-minified WPKG
distribution and if the problem goes away do not file a bug for such issues.

One might save already a lot of size if a minifier is used which is just
removing the comments in the file. As I have added a lot of documentation in
comments this might already reduce the size to ~150kB, so further code
minification like variable rename etc. might not add a lot more gain. Removing
comments is pretty safe operation. Again, I don't think saving a couple of kB
justifies potential problems in WPKG use case.


I am not closing the issue yet as I want to keep the discussion open. Asking
for more opinion on this "issue". Does anybody else feel that size of wpkg.js
matters in their environment?

-- 
Configure bugmail: http://bugzilla.wpkg.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.