Home / ubuntu / Questions / 285482
Accepted
Aditya
Asked: 2013-04-26 09:43:23 +0800 CST

How to reduce the size of apt-get update list on a Development Release once it has been released as Stable?

  • 772

I have been using the development version of 13.04 since early March. In development release, the general behavior of running the sudo apt-get update command is to download the list files completely if they have been changed. Whereas, if one uses the Stable version of Ubuntu, the command just downloads the diff of the list files if they have been changed (rsync style).

Basically, in development versions sudo apt-get update downloads about 17-18 MB of data; whereas in stable versions only about 1 MB of data gets downloaded (except for the initial run).

Now since 13.04 has been released as Stable, I would like to change the behavior of sudo apt-get update to mimic its normal behavior in a Stable release. How do I make this happen?

A similar question but for the Stable Release was earlier asked on Ask Ubuntu:

However, it was a bug in 12.04 which got fixed later on. But the answers given there are not feasible in my case. For Ubuntu+1 its not a bug, its a general behavior of Development releases. Anyways, I would go through the answers provided there:

  • The answer by John S Gruber asks to touch the relevant files before I run the command (sudo apt-get update) everytime. It's definitely not feasible in my case. That answer was okay for that question since it was a bug in 12.04 and it provided a temporary fix in that situation.

  • The answer by Chan-Ho Suh asks to change the update mirrors. This wouldn't solve the problem in a Development Release.

  • The answer by Anwar was also a temporary fix for that situation. It assumes I would run sudo apt-get update command only after adding a PPA and so asks to disable the other repositories. This is not feasible in my case since I obviously want to update all the repositories. I don't necessarily run this command only after adding a PPA. I run this command on a daily basis to install any updates released by Canonical. I use sudo apt-get update as a subsitute for Software Center.

So here it is. How do I restore the normal behavior of sudo apt-get update command to download only the diff of the changed files? Hopefully, the same solution would work for Software Center as well, since Software Center also consumes similar amount of bandwidth if I update using it.

I definitely do not want to re-install 13.04 just to restore this behavior.

Output of sudo apt-get update in case it matters:

$ sudo apt-get update
Hit http://dl.google.com stable Release.gpg                                    
Hit http://dl.google.com stable Release                                        
Hit http://ppa.launchpad.net raring Release.gpg                                
Hit http://dl.google.com stable/main i386 Packages                             
Hit http://linux.dropbox.com precise Release.gpg                               
Hit http://ppa.launchpad.net raring Release.gpg                      
Get:1 http://mirror.picosecond.org raring Release.gpg [933 B]        
Hit http://ppa.launchpad.net raring Release                                    
Hit http://linux.dropbox.com precise Release                                   
Get:2 http://mirror.picosecond.org raring-updates Release.gpg [933 B]          
Hit http://ppa.launchpad.net raring Release                                    
Hit http://linux.dropbox.com precise/main i386 Packages                        
Hit http://ppa.launchpad.net raring/main i386 Packages                         
Get:3 http://mirror.picosecond.org raring-backports Release.gpg [933 B]
Ign http://dl.google.com stable/main Translation-en_IN                         
Ign http://dl.google.com stable/main Translation-en                            
Hit http://mirror.picosecond.org raring-security Release.gpg          
Get:4 http://mirror.picosecond.org raring Release [40.8 kB]           
Hit http://ppa.launchpad.net raring/main i386 Packages                       
Get:5 http://mirror.picosecond.org raring-updates Release [40.8 kB]            
Get:6 http://mirror.picosecond.org raring-backports Release [40.8 kB]          
Ign http://linux.dropbox.com precise/main Translation-en_IN                    
Ign http://linux.dropbox.com precise/main Translation-en                       
Hit http://mirror.picosecond.org raring-security Release                       
Get:7 http://mirror.picosecond.org raring/main Sources [963 kB]                
Ign http://ppa.launchpad.net raring/main Translation-en_IN                     
Ign http://ppa.launchpad.net raring/main Translation-en                        
Ign http://ppa.launchpad.net raring/main Translation-en_IN                     
Ign http://ppa.launchpad.net raring/main Translation-en                        
Get:8 http://mirror.picosecond.org raring/restricted Sources [5,987 B]         
Get:9 http://mirror.picosecond.org raring/universe Sources [5,838 kB]          
Get:10 http://mirror.picosecond.org raring/multiverse Sources [171 kB]         
Get:11 http://mirror.picosecond.org raring/main i386 Packages [1,168 kB]       
Get:12 http://mirror.picosecond.org raring/restricted i386 Packages [9,623 B]  
Get:13 http://mirror.picosecond.org raring/universe i386 Packages [5,405 kB]   
Get:14 http://mirror.picosecond.org raring/multiverse i386 Packages [131 kB]   
Hit http://mirror.picosecond.org raring/main Translation-en                    
Hit http://mirror.picosecond.org raring/multiverse Translation-en              
Hit http://mirror.picosecond.org raring/restricted Translation-en              
Get:15 http://mirror.picosecond.org raring/universe Translation-en [3,736 kB]  
Get:16 http://mirror.picosecond.org raring-updates/main Sources [1,616 B]      
Get:17 http://mirror.picosecond.org raring-updates/restricted Sources [14 B]   
Get:18 http://mirror.picosecond.org raring-updates/universe Sources [14 B]     
Get:19 http://mirror.picosecond.org raring-updates/multiverse Sources [14 B]   
Get:20 http://mirror.picosecond.org raring-updates/main i386 Packages [3,331 B]
Get:21 http://mirror.picosecond.org raring-updates/restricted i386 Packages [14 B]
Get:22 http://mirror.picosecond.org raring-updates/universe i386 Packages [1,078 B]
Get:23 http://mirror.picosecond.org raring-updates/multiverse i386 Packages [14 B]
Hit http://mirror.picosecond.org raring-updates/main Translation-en            
Hit http://mirror.picosecond.org raring-updates/multiverse Translation-en      
Hit http://mirror.picosecond.org raring-updates/restricted Translation-en      
Hit http://mirror.picosecond.org raring-updates/universe Translation-en        
Get:24 http://mirror.picosecond.org raring-backports/main Sources [14 B]       
Get:25 http://mirror.picosecond.org raring-backports/restricted Sources [14 B] 
Get:26 http://mirror.picosecond.org raring-backports/universe Sources [738 B]  
Get:27 http://mirror.picosecond.org raring-backports/multiverse Sources [14 B] 
Get:28 http://mirror.picosecond.org raring-backports/main i386 Packages [14 B] 
Get:29 http://mirror.picosecond.org raring-backports/restricted i386 Packages [14 B]
Get:30 http://mirror.picosecond.org raring-backports/universe i386 Packages [508 B]
Get:31 http://mirror.picosecond.org raring-backports/multiverse i386 Packages [14 B]
Hit http://mirror.picosecond.org raring-backports/main Translation-en          
Hit http://mirror.picosecond.org raring-backports/multiverse Translation-en    
Hit http://mirror.picosecond.org raring-backports/restricted Translation-en    
Hit http://mirror.picosecond.org raring-backports/universe Translation-en      
Hit http://mirror.picosecond.org raring-security/main Sources                  
Hit http://mirror.picosecond.org raring-security/restricted Sources            
Hit http://mirror.picosecond.org raring-security/universe Sources              
Hit http://mirror.picosecond.org raring-security/multiverse Sources            
Hit http://mirror.picosecond.org raring-security/main i386 Packages            
Hit http://mirror.picosecond.org raring-security/restricted i386 Packages      
Hit http://mirror.picosecond.org raring-security/universe i386 Packages        
Hit http://mirror.picosecond.org raring-security/multiverse i386 Packages      
Hit http://mirror.picosecond.org raring-security/main Translation-en           
Hit http://mirror.picosecond.org raring-security/multiverse Translation-en     
Hit http://mirror.picosecond.org raring-security/restricted Translation-en
Hit http://mirror.picosecond.org raring-security/universe Translation-en
Ign http://mirror.picosecond.org raring/main Translation-en_IN
Ign http://mirror.picosecond.org raring/multiverse Translation-en_IN
Ign http://mirror.picosecond.org raring/restricted Translation-en_IN
Ign http://mirror.picosecond.org raring/universe Translation-en_IN
Ign http://mirror.picosecond.org raring-updates/main Translation-en_IN
Ign http://mirror.picosecond.org raring-updates/multiverse Translation-en_IN
Ign http://mirror.picosecond.org raring-updates/restricted Translation-en_IN
Ign http://mirror.picosecond.org raring-updates/universe Translation-en_IN
Ign http://mirror.picosecond.org raring-backports/main Translation-en_IN
Ign http://mirror.picosecond.org raring-backports/multiverse Translation-en_IN
Ign http://mirror.picosecond.org raring-backports/restricted Translation-en_IN
Ign http://mirror.picosecond.org raring-backports/universe Translation-en_IN
Ign http://mirror.picosecond.org raring-security/main Translation-en_IN
Ign http://mirror.picosecond.org raring-security/multiverse Translation-en_IN
Ign http://mirror.picosecond.org raring-security/restricted Translation-en_IN
Ign http://mirror.picosecond.org raring-security/universe Translation-en_IN
Fetched 17.6 MB in 3min 46s (77.7 kB/s)
Reading package lists... Done

Contents of /etc/apt/sources.list in case it matters:

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to

# newer versions of the distribution.
deb http://mirror.picosecond.org/ubuntu/ raring main restricted
deb-src http://mirror.picosecond.org/ubuntu/ raring main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://mirror.picosecond.org/ubuntu/ raring-updates main restricted
deb-src http://mirror.picosecond.org/ubuntu/ raring-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://mirror.picosecond.org/ubuntu/ raring universe
deb-src http://mirror.picosecond.org/ubuntu/ raring universe
deb http://mirror.picosecond.org/ubuntu/ raring-updates universe
deb-src http://mirror.picosecond.org/ubuntu/ raring-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
## team, and may not be under a free licence. Please satisfy yourself as to 
## your rights to use the software. Also, please note that software in 
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://mirror.picosecond.org/ubuntu/ raring multiverse
deb-src http://mirror.picosecond.org/ubuntu/ raring multiverse
deb http://mirror.picosecond.org/ubuntu/ raring-updates multiverse
deb-src http://mirror.picosecond.org/ubuntu/ raring-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://mirror.picosecond.org/ubuntu/ raring-backports main restricted universe multiverse
deb-src http://mirror.picosecond.org/ubuntu/ raring-backports main restricted universe multiverse

deb http://mirror.picosecond.org/ubuntu/ raring-security main restricted
deb-src http://mirror.picosecond.org/ubuntu/ raring-security main restricted
deb http://mirror.picosecond.org/ubuntu/ raring-security universe
deb-src http://mirror.picosecond.org/ubuntu/ raring-security universe
deb http://mirror.picosecond.org/ubuntu/ raring-security multiverse
deb-src http://mirror.picosecond.org/ubuntu/ raring-security multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu raring partner
# deb-src http://archive.canonical.com/ubuntu raring partner

## Uncomment the following two lines to add software from Ubuntu's
## 'extras' repository.
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
# deb http://extras.ubuntu.com/ubuntu raring main
# deb-src http://extras.ubuntu.com/ubuntu raring main
apt

2 Answers

  1. Best Answer

    apt does not actually do anything different itself between a development release and a stable release.

    However, the release itself stops changing, and so the size of the package list that apt has to download gets much smaller.

    Let's take Raring as an example.

    During development, the raring-updates and raring-security pockets remained empty, but the raring pocket updated on a regular (half-hourly) basis.

    After release, the raring pocket stopped changing. Updates now get uploaded as new packages entering the raring-updates and raring-security pockets.

    From apt's point of view, the raring package list was downloaded in full every time apt-get update was run during development (unless it was called two times or more within the half hour window in between upstream updates). This is a large list, so it took some time.

    Now that Raring is released, the raring package list no longer changes. The raring-updates and raring-security package lists do change, but these are much smaller, since they only contain packages that have "changed" in Raring since Raring's release.

    I can see this behaviour in your apt output in your question. It looks just as expected: the raring pocket is just getting Hit with no download; the Get against raring-updates and raring-security corresponds to actual downloads. Edit: looking again, it looks like some files are being unnecessarily re-downloaded; see my other answer.

    If this is too slow for you, consider using a faster mirror, or dropping the rather large list of PPAs you have there. You could also drop the deb-src lines from sources.list if you don't need the source available to apt-get.

    I don't believe that you'd see behaviour any different if you had installed this release directly.

    • 3

  2. Another possibility is that you are using a mirror that does not preserve timestamps, or a corrupt local apt cache, causing apt to re-download files every time that have not actually changed in the mirror. Try this answer to clear your local cache, and switch to a mirror that does preserve timestamps.

    • 1