Home / ubuntu / Questions / 1275269
In Process
sancho.s ReinstateMonicaCellio
Asked: 2020-09-16 02:41:30 +0800 CST

apt-get install behind proxy - Failed to fetch

  • 772

This is yet another question regarding the error message E: Failed to fetch ... (example in Note 2 below). What makes this question different (I guess) from the others I have seen is the following.

I am running Ubuntu 18.04LTS, behind a proxy. I had a lot of packages with available updates (more than 200, I guess).

    I could update many of the packages with no problem, and I ended up with 48 that I cannot install.

All of the packages not installed belong to bionic-updates/universe or bionic-updates/main. And most are related to java, jdk, linux, systemd. The complete list is in Note 1 below. But there are packages from bionic-updates/universe and bionic-updates/main that I can install (I just installed qemu and bsdutils).

How is it possible that some of the packages can be installed and some cannot?
What is a possible solution (other than connecting the PC without a proxy)?

Notes:

  1. Complete list of packages not installed

    ant ant-optional apport apport-gtk fwupd fwupdate fwupdate-signed gettext gettext-base jarwrapper junit4 libasm-java libcommons-collections3-java libcommons-compress-java libcommons-httpclient-java libdb5.3 libdb5.3-java libdb5.3-java-jni libecj-java libequinox-osgi-java libhsqldb1.8.0-java libjetty9-java libnss-systemd libobjenesis-java libpam-systemd libservlet3.1-java libsystemd0 libtomcat8-java linux-generic linux-headers-generic linux-image-generic openjdk-11-jdk openjdk-11-jdk-headless openjdk-11-jre openjdk-11-jre-headless openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless python-pip-whl python3-pip snapd systemd

This was obtained with

$ sudo apt list --upgradable 2> /dev/null | sed 's/\/.*//' | tail +2 | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g'

To find which repo each one belonged, I used (printf "%30s"... helps visualizing in columns)

$ for p in $(sudo apt list --upgradable 2> /dev/null | sed 's/\/.*//' | tail +2 | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g') ; do printf "%30s" $p ; apt policy $p 2> /dev/null | tail +6 | head -1 ; done

This generates output like

                       ant        500 http://us.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages
  1. Sample error
    $ sudo apt-get install apport
    Reading package lists... Done
    Building dependency tree       
    Reading state information... Done
    The following packages will be upgraded:
      apport
    1 upgraded, 0 newly installed, 0 to remove and 46 not upgraded.
    Need to get 125 kB of archives.
    After this operation, 20,5 kB of additional disk space will be used.
    Ign:1 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 apport all 2.20.9-0ubuntu7.17
    Err:1 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 apport all 2.20.9-0ubuntu7.17
      403  URLBlockDownload [IP: xxx.xxx.xxx3.xxx 80]
    E: Failed to fetch http://us.archive.ubuntu.com/ubuntu/pool/main/a/apport/apport_2.20.9-0ubuntu7.17_all.deb  403  URLBlockDownload [IP: 10.1.33.101 80]
    E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  1. EDIT: output "requested" in this answer is here.
    $ find /etc/apt/ -name "*.list" -exec grep -iq http {} \; -exec egrep -v '^#|^$' {} \; | uniq 
    deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main
    deb http://us.archive.ubuntu.com/ubuntu/ bionic main restricted
    deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates main restricted
    deb http://us.archive.ubuntu.com/ubuntu/ bionic universe
    deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates universe
    deb http://us.archive.ubuntu.com/ubuntu/ bionic multiverse
    deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates multiverse
    deb http://us.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse
    deb http://security.ubuntu.com/ubuntu bionic-security main restricted
    deb http://security.ubuntu.com/ubuntu bionic-security universe
    deb http://security.ubuntu.com/ubuntu bionic-security multiverse
    deb https://apt.kitware.com/ubuntu/ bionic main
updates apt proxy 18.04

3 Answers

  1. It looks like enterprise firewall is blocking certain urls of apt, that's why you are not able to download packages via apt. You need to get all URLs white listed in firewall mentioned in source list. You need to have admin access of firewall or kindly contact network administrator.

    As Alternative option, try to connect system directly with another internet source like USB Dongle or Mobile.

    • 2

  2. This is an effort to give a more solid basis to the suggestion in this answer. It mentions that firewall filtering is the culprit, and that it is based on the full URI used for each package, including the package name itself. That might explain why some packages from a given repo can be installed and some others cannot. To test this, I have obtained the URIs for all packages that I cannot install with --print-uris (see full contrived command line in Note 1 below), and for package yudit-common which I don't have, and I could later install. Then I tried wgetting those URIs.

    For each one of the packages that I was not able to install, the .deb file for either that package or one of its dependencies produced

    $ wget http://us.archive.ubuntu.com/ubuntu/pool/universe/a/ant/ant-optional_1.10.5-3~18.04_all.deb
--2020-09-22 10:18:35--  http://us.archive.ubuntu.com/ubuntu/pool/universe/a/ant/ant-optional_1.10.5-3~18.04_all.deb
Resolving proxy-myserver (proxy-myserver)... xx.xx.xx.xx
Connecting to proxy-myserver (proxy-myserver)|xx.xx.xx.xx|:xx... connected.
Proxy request sent, awaiting response... 403 URLBlockDownload
2020-09-22 10:18:35 ERROR 403: URLBlockDownload.

    This would more strongly confirm the proposal.

    Notes:

    1. Obtaining full URIs for packages (and also trying wget here)
        for p in $(sudo apt list --upgradable 2> /dev/null | sed 's/\/.*//' | tail +2 | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g') ; do for uri in $(apt install --print-uris $p 2> /dev/null | sed -n '/^After this operation/ { :a; n; p; ba; }' | awk '{ print substr($1,2,length($1)-2) }') ; echo "uri=${uri}" ; wget ${uri} ; done ; done
    • 1

  3. Proxy server can be set up different ways. One way is to allow access to specific sites is by wildcard domain or vice versa. If admin or environment for this is even more paranoid it is possible make exceptions even more granual allowing access to specific sites by regex formulas to specific URI-s.

    What is the output of apt-get update does it have any errors?

    • 0