Hidden Secrets of Dockerfile

Have you ever faced an issue in which while building the docker image, even though you have mentioned “apt-get -y update” the package you are trying to install is not found in the repository?

Here, in this blog, we are going to discuss two common scenarios that I faced along with the root cause of this problem. Let’s take a look.

First, it’s important to remember that valid intermediate images are not built again. They are loaded from cache. Dockerfile cache is considered valid if the build command gives a 0 return code.


1. Consider the following Dockerfile.

From ubuntu
RUN apt-get -y update RUN apt-get install -y python-pip

Now when we build it:

docker build -t test:image1 .

Suppose while building the above Dockerfile we created an intermediary image with the directive “RUN apt-get update”.

This intermediate image will be used for all the subsequent docker builds.

Let’s say a couple of months down the line, there is a new package-manager release of python-pip which is required by the application. Now, even though in Dockerfile we have mentioned “RUN apt-get update”, the apt repo will not get updated because that image will be taken from the cache. Hence the latest version of python-pip won’t be installed.

2. Our story is not over yet, consider the following.

Before building the below Dockerfile, turn your internet connection off.

From ubuntu
RUN apt-get -y update ; echo "status is: $?" RUN apt-get install -y python-pip

Now when we build it:

Docker build -t test:image2 .

Output:

Err http://security.debian.org stable/updates Release.gpg 
Could not resolve 'security.debian.org' 
Hit http://192.168.1.100 stable Release.gpg 
Hit http://192.168.1.100 stable Release
Hit http://192.168.1.100 stable/main i386 Packages 
Hit http://192.168.1.100 stable/contrib i386 Packages
Hit http://192.168.1.100 stable/non-free i386 Packages
Ign http://192.168.1.100 stable/contrib Translation-en
Ign http://192.168.1.100 stable/main Translation-en
Ign http://192.168.1.100 stable/non-free Translation-en
Reading package lists... Done 
W: Failed to fetch http://security.debian.org/dists/stable/updates/Release.gpg 
W: Some index files failed to download. They have been ignored, or old ones used instead.


status is: 0


 Aside from the ADD and COPY commands, cache checking does not look at any other file in the container to determine a cache match. For example, when processing a RUN apt-get -y update command, the files updated in the container are not examined to determine if a cache hit exists. In such a case, just the command string itself is used to find a match.

As we can see above, “apt-get update” throws a “WARN” and not an “ERROR” and the return code is 0, which is a successful response. Since the response code is zero hence this image will get cached.

The problem comes when we re-build the Dockerfile with our internet connection on, Docker will still take the “apt-get -y update” image from the cache and subsequent “RUN apt-get install -y python-pip” will fail because “apt-get -y update” didn’t trigger successfully in the last run.

Solution:

Try building the image again with the “no-cache” option.  This will force the rebuilding of layers already available.

Docker build --no-cache -t test:image3 . 

Conclusion :
Once an image is successfully built, all the intermediate images will be kept and reused. A valid docker intermediate image will not be built again unless we pass the “no-cache” option or delete the existing image.

Image Resource

Opstree is an End to End DevOps solution provider

CONTACT US

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s