python not - How to install packages using pip according to the requirements.txt file from a local directory?




pip.conf text (8)

Here is the problem

I have a requirements.txt that looks like:

BeautifulSoup==3.2.0
Django==1.3
Fabric==1.2.0
Jinja2==2.5.5
PyYAML==3.09
Pygments==1.4
SQLAlchemy==0.7.1
South==0.7.3
amqplib==0.6.1
anyjson==0.3
...

I have a local archive directory containing all the packages + others.

I have created a new virtualenv with

bin/virtualenv testing

upon activating it, I tried to install the packages according to requirements.txt from the local archive directory.

source bin/activate
pip install -r /path/to/requirements.txt -f file:///path/to/archive/

I got some output that seems to indicate that the installation is fine

Downloading/unpacking Fabric==1.2.0 (from -r ../testing/requirements.txt (line 3))
  Running setup.py egg_info for package Fabric
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no files found matching 'fabfile.py'
Downloading/unpacking South==0.7.3 (from -r ../testing/requirements.txt (line 8))
  Running setup.py egg_info for package South
....

But later check revealed none of the package is installed properly. I cannot import the package, and none is found in the site-packages directory of my virtualenv. So what went wrong?


Answers

I work with a lot of systems that have been mucked by developers "following directions they found on the internet". It is extremely common that your pip and your python are not looking at the same paths/site-packages. For this reason, when I encounter oddness I start by doing this:

$ python -c 'import sys; print(sys.path)'
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages']

$ pip --version
pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)

That is a happy system.

Below is an unhappy system. (Or at least it's a blissfully ignorant system that causes others to be unhappy.)

$ pip --version
pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6)

$ python -c 'import sys; print(sys.path)'
['', '/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/site-packages']

$ which pip pip2 pip3
/usr/local/bin/pip
/usr/local/bin/pip3

It is unhappy because pip is (python3.6 and) using /usr/local/lib/python3.6/site-packages while python is (python2.7 and) using /usr/local/lib/python2.7/site-packages

When I want to make sure I'm installing requirements to the right python, I do this:

$ which -a python python2 python3
/usr/local/bin/python
/usr/bin/python
/usr/local/bin/python2
/usr/local/bin/python3

$ /usr/bin/python -m pip install -r requirements.txt

You've heard, "If it ain't broke, don't try to fix it." The DevOps version of that is, "If you didn't break it and you can work around it, don't try to fix it."


Installing requirements.txt file inside virtual env with python 3:

I had the same issue. I was trying to install requirements.txt file inside a virtual environament. I found the solution.

Initially, I created my virtual env in this way:

virtualenv -p python3 myenv

Activate the environment using:

source myenv/bin/activate

Now I installed the requirements.txt using:

pip3 install -r requirements.txt

Installation was successful and I was able to import the modules.


Often, you will want a fast install from local archives, without probing PyPI.

First, download the archives that fulfill your requirements:

$ pip install --download <DIR> -r requirements.txt

Then, install using –find-links and –no-index:

$ pip install --no-index --find-links=[file://]<DIR> -r requirements.txt

I had a similar problem. I tried this:

pip install -U -r requirements.txt 

(-U = update if it had already installed)

But the problem continued. I realized that some of generic libraries for development were missed.

sudo apt-get install libtiff5-dev libjpeg8-dev zlib1g-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk

I don't know if this would help you.


pip install -r requirements.txt

For further details please check the help option.

pip install --help

We can find the option '-r'

-r, --requirement Install from the given requirements file. This option can be used multiple times.

Further information on some commonly used pip install options: (This is the help option on pip install command)

Also the above is the complete set of options. Please use pip install --help for complete list of options.


This works for me:

$ pip install -r requirements.txt --no-index --find-links file:///tmp/packages

--no-index - Ignore package index (only looking at --find-links URLs instead).

-f, --find-links <URL> - If a URL or path to an html file, then parse for links to archives. If a local path or file:// URL that's a directory, then look for archives in the directory listing.


I've read the above, realize this is an old question, but it's totally unresolved and still at the top of my google search results so here's an answer that works for everyone:

pip install -r /path/to/requirements.txt

You can get a list of packages and their versions, e.g.:

pip list

Output :

appdirs (1.4.3)
BeautifulSoup (3.2.
beautifulsoup4 (4.6
certifi (2017.4.17)
chardet (3.0.4)
cookies (2.2.1)
dj-database-url (0.
Django (1.10.4)
django-allauth (0.3
django-filter (1.0.
django-haystack (2.
django-oauth-toolki
django-recaptcha (1
djangorestframework
djangorestframework
funcsigs (1.0.2)
gunicorn (19.7.1)
idna (2.5)
Jinja2 (2.9.6)
Markdown (2.6.8)
MarkupSafe (1.0)
mock (2.0.0)
MySQL-python (1.2.5
numpy (1.13.1)
oauthlib (2.0.2)
packaging (16.8)
pandas (0.20.3)
pbr (3.1.1)
pep8 (1.7.0)
pip (9.0.1)
psycopg2 (2.7.1)
PyJWT (1.5.2)
pyparsing (2.2.0)
python-dateutil (2.
python-decouple (3.
python-openid (2.2.
pytz (2017.2)
requests (2.18.1)
requests-oauthlib (
responses (0.5.1)
setuptools (35.0.1)
six (1.10.0)
social-auth-app-dja
social-auth-core (1
urllib3 (1.21.1)
virtualenv (15.1.0)
wheel (0.29.0)
whitenoise (3.3.0)




python virtualenv pip