wget: Proxies
1
1 8.1 Proxies
1 ===========
1
1 “Proxies” are special-purpose HTTP servers designed to transfer data
1 from remote servers to local clients. One typical use of proxies is
1 lightening network load for users behind a slow connection. This is
1 achieved by channeling all HTTP and FTP requests through the proxy which
1 caches the transferred data. When a cached resource is requested again,
1 proxy will return the data from cache. Another use for proxies is for
1 companies that separate (for security reasons) their internal networks
1 from the rest of Internet. In order to obtain information from the Web,
1 their users connect and retrieve remote data using an authorized proxy.
1
1 Wget supports proxies for both HTTP and FTP retrievals. The standard
1 way to specify proxy location, which Wget recognizes, is using the
1 following environment variables:
1
1 ‘http_proxy’
1 ‘https_proxy’
1 If set, the ‘http_proxy’ and ‘https_proxy’ variables should contain
1 the URLs of the proxies for HTTP and HTTPS connections
1 respectively.
1
1 ‘ftp_proxy’
1 This variable should contain the URL of the proxy for FTP
1 connections. It is quite common that ‘http_proxy’ and ‘ftp_proxy’
1 are set to the same URL.
1
1 ‘no_proxy’
1 This variable should contain a comma-separated list of domain
1 extensions proxy should _not_ be used for. For instance, if the
1 value of ‘no_proxy’ is ‘.mit.edu’, proxy will not be used to
1 retrieve documents from MIT.
1
1 In addition to the environment variables, proxy location and settings
1 may be specified from within Wget itself.
1
1 ‘--no-proxy’
1 ‘proxy = on/off’
1 This option and the corresponding command may be used to suppress
1 the use of proxy, even if the appropriate environment variables are
1 set.
1
1 ‘http_proxy = URL’
1 ‘https_proxy = URL’
1 ‘ftp_proxy = URL’
1 ‘no_proxy = STRING’
1 These startup file variables allow you to override the proxy
1 settings specified by the environment.
1
1 Some proxy servers require authorization to enable you to use them.
1 The authorization consists of “username” and “password”, which must be
1 sent by Wget. As with HTTP authorization, several authentication
1 schemes exist. For proxy authorization only the ‘Basic’ authentication
1 scheme is currently implemented.
1
1 You may specify your username and password either through the proxy
1 URL or through the command-line options. Assuming that the company’s
1 proxy is located at ‘proxy.company.com’ at port 8001, a proxy URL
1 location containing authorization data might look like this:
1
1 http://hniksic:mypassword@proxy.company.com:8001/
1
1 Alternatively, you may use the ‘proxy-user’ and ‘proxy-password’
1 options, and the equivalent ‘.wgetrc’ settings ‘proxy_user’ and
1 ‘proxy_password’ to set the proxy username and password.
1