wget: Proxies

1 
1 8.1 Proxies
1 ===========
1 
1 “Proxies” are special-purpose HTTP servers designed to transfer data
1 from remote servers to local clients.  One typical use of proxies is
1 lightening network load for users behind a slow connection.  This is
1 achieved by channeling all HTTP and FTP requests through the proxy which
1 caches the transferred data.  When a cached resource is requested again,
1 proxy will return the data from cache.  Another use for proxies is for
1 companies that separate (for security reasons) their internal networks
1 from the rest of Internet.  In order to obtain information from the Web,
1 their users connect and retrieve remote data using an authorized proxy.
1 
1    Wget supports proxies for both HTTP and FTP retrievals.  The standard
1 way to specify proxy location, which Wget recognizes, is using the
1 following environment variables:
1 
1 ‘http_proxy’
1 ‘https_proxy’
1      If set, the ‘http_proxy’ and ‘https_proxy’ variables should contain
1      the URLs of the proxies for HTTP and HTTPS connections
1      respectively.
1 
1 ‘ftp_proxy’
1      This variable should contain the URL of the proxy for FTP
1      connections.  It is quite common that ‘http_proxy’ and ‘ftp_proxy’
1      are set to the same URL.
1 
1 ‘no_proxy’
1      This variable should contain a comma-separated list of domain
1      extensions proxy should _not_ be used for.  For instance, if the
1      value of ‘no_proxy’ is ‘.mit.edu’, proxy will not be used to
1      retrieve documents from MIT.
1 
1    In addition to the environment variables, proxy location and settings
1 may be specified from within Wget itself.
1 
1 ‘--no-proxy’
1 ‘proxy = on/off’
1      This option and the corresponding command may be used to suppress
1      the use of proxy, even if the appropriate environment variables are
1      set.
1 
1 ‘http_proxy = URL’
1 ‘https_proxy = URL’
1 ‘ftp_proxy = URL’
1 ‘no_proxy = STRING’
1      These startup file variables allow you to override the proxy
1      settings specified by the environment.
1 
1    Some proxy servers require authorization to enable you to use them.
1 The authorization consists of “username” and “password”, which must be
1 sent by Wget.  As with HTTP authorization, several authentication
1 schemes exist.  For proxy authorization only the ‘Basic’ authentication
1 scheme is currently implemented.
1 
1    You may specify your username and password either through the proxy
1 URL or through the command-line options.  Assuming that the company’s
1 proxy is located at ‘proxy.company.com’ at port 8001, a proxy URL
1 location containing authorization data might look like this:
1 
1      http://hniksic:mypassword@proxy.company.com:8001/
1 
1    Alternatively, you may use the ‘proxy-user’ and ‘proxy-password’
1 options, and the equivalent ‘.wgetrc’ settings ‘proxy_user’ and
1 ‘proxy_password’ to set the proxy username and password.
1