Metadata-Version: 2.0
Name: crawlib
Version: 0.0.10
Summary: tool set for crawler project.
Home-page: https://github.com/MacHu-GWU/
Author: Sanhe Hu
Author-email: husanhe@gmail.com
License: MIT
Download-URL: https://github.com/MacHu-GWU//tarball/2017-11-08
Description-Content-Type: UNKNOWN
Platform: Windows
Platform: MacOS
Platform: Unix
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Requires-Dist: six
Requires-Dist: chardet
Requires-Dist: requests
Requires-Dist: selenium
Requires-Dist: beautifulsoup4
Requires-Dist: attrs
Requires-Dist: fake-useragent
Requires-Dist: diskcache

.. image:: https://travis-ci.org/MacHu-GWU/crawlib-project.svg?branch=master
    :target: https://travis-ci.org/MacHu-GWU/crawlib-project?branch=master

.. image:: https://codecov.io/gh/MacHu-GWU/crawlib-project/branch/master/graph/badge.svg
  :target: https://codecov.io/gh/MacHu-GWU/crawlib-project

.. image:: https://img.shields.io/pypi/v/crawlib.svg
    :target: https://pypi.python.org/pypi/crawlib

.. image:: https://img.shields.io/pypi/l/crawlib.svg
    :target: https://pypi.python.org/pypi/crawlib

.. image:: https://img.shields.io/pypi/pyversions/crawlib.svg
    :target: https://pypi.python.org/pypi/crawlib

.. image:: https://img.shields.io/badge/Star_Me_on_GitHub!--None.svg?style=social
    :target: https://github.com/MacHu-GWU/crawlib-project


Welcome to ``crawlib`` Documentation
==============================================================================
``crawlib`` is a crawler project building block to simplify:

1. url encoding
2. html parse
3. error handling
4. download html and file


Quick Links
------------------------------------------------------------------------------

- .. image:: https://img.shields.io/badge/Link-Document-red.svg
      :target: http://www.wbh-doc.com.s3.amazonaws.com/crawlib/index.html

- .. image:: https://img.shields.io/badge/Link-API_Reference_and_Source_Code-red.svg
      :target: API reference and source code <http://www.wbh-doc.com.s3.amazonaws.com/crawlib/py-modindex.html

- .. image:: https://img.shields.io/badge/Link-Install-red.svg
      :target: `install`_

- .. image:: https://img.shields.io/badge/Link-GitHub-blue.svg
      :target: https://github.com/MacHu-GWU/crawlib-project

- .. image:: https://img.shields.io/badge/Link-Submit_Issue_and_Feature_Request-blue.svg
      :target: https://github.com/MacHu-GWU/crawlib-project/issues

- .. image:: https://img.shields.io/badge/Link-Download-blue.svg
      :target: https://pypi.python.org/pypi/crawlib#downloads


.. _install:

Install
------------------------------------------------------------------------------

``crawlib`` is released on PyPI, so all you need is:

.. code-block:: console

    $ pip install crawlib

To upgrade to latest version:

.. code-block:: console

    $ pip install --upgrade crawlib

