Metadata-Version: 1.1
Name: spyda
Version: 0.0.2
Summary: Spyda - Python Spider Tool and Library
Home-page: https://bitbucket.org/prologic/spyda
Author: James Mills
Author-email: James Mills, j dot mills at griffith dot edu dot au
License: MIT
Download-URL: https://bitbucket.org/prologic/spyda/downloads/
Description: .. _Python Programming Language: http://www.python.org/
        .. _Python Standard Library: http://docs.python.org/library/
        .. _restclient: http://pypi.python.org/pypi/restclient
        .. _cssselect: http://pypi.python.org/pypi/cssselect
        .. _lxml: http://pypi.python.org/pypi/lxml/3.0.2
        .. _url: http://pypi.python.org/pypi/url
        .. _nltk: https://pypi.python.org/pypi/nltk
        .. _calais: https://bitbucket.org/prologic/calais
        .. _BeautifulSoup: https://pypi.python.org/pypi/BeautifulSoup
        .. _Griffith University: http://www.griffith.edu.au/
        .. _Project Website: http://bitbucket.org/prologic/spyda
        .. _PyPi Page: http://pypi.python.org/pypi/spyda
        .. _Read the Docs: http://spyda.readthedocs.org/en/latest/
        .. _Downloads Page: https://bitbucket.org/prologic/spyda/downloads
        
        
        spyda is a simple tool and library written in the `Python Programming Language`_ to crawl a given url whilst allowing you to restrict results to a specified
        domain and optionally also perform pattern matching against URLs crawled. spyda will report on any URLs it was unable to crawl along with their status code
        and store successfully crawled links and their content in a directory structure that matches the domain and URLs searched.
        
        spyda was developed at `Griffith University`_ as a tool and library to assist with web crawling tasks and data extraction and has been used to help
        match researcher names against publications as well as extract data and links from external sources of data.
        
        - Visit the `Project Website`_
        - `Read the Docs`_
        - Download it from the `Downloads Page`_
        
        .. image:: https://pypip.in/v/spyda/badge.png
           :target: https://crate.io/packages/spyda/
           :alt: Latest PyPI version
        
        .. image:: https://pypip.in/d/spyda/badge.png
           :target: https://crate.io/packages/spyda/
           :alt: Number of PyPI downloads
        
        .. image:: https://jenkins.shiningpanda-ci.com/prologic/job/spyda/badge/icon
           :target: https://jenkins.shiningpanda-ci.com/prologic/job/spyda/
           :alt: Build Status
        
        
        Requirements
        ------------
        
        - `restclient`_
        - `cssselect`_
        - `lxml`_
        - `url`_
        - `nltk`_
        - `calais`_
        - `BeautifulSoup`_
        
        spyda also comes basic documentation and a full comprehensive unit test suite which require the following:
        
        To build the docs:
        
        - `sphinx <https://pypi.python.org/pypi/Sphinx>`_
        - `sphinxcontrib-bitbucket <https://pypi.python.org/pypi/sphinxcontrib-bitbucket>`_
        
        To run the unit tests:
        
        - `pytest <https://pypi.python.org/pypi/pytest>`_
        - `circuits <https://pypi.python.org/pypi/circuits>`_
        
        
        Installation
        ------------
        
        The simplest and recommended way to install spyda is with pip.
        You may install the latest stable release from PyPI with pip::
        
            > pip install spyda
        
        If you do not have pip, you may use easy_install::
        
            > easy_install spyda
        
        Alternatively, you may download the source package from the
        `PyPI Page`_ or the `Downloads page`_ on the `Project Website`_;
        extract it and install using::
        
            > python setup.py install
        
        You can also install the
        `latest-development version <https://bitbucket.org/prologic/spyda/get/tip.tar.gz#egg=spyda-dev>`_ by using ``pip`` or ``easy_install``::
            
            > pip install spyda==dev
        
        or::
            
            > easy_install spyda==dev
        
        
        For further information see the `spyda documentation <http://spyda.readthedocs.org/>`_.
        
        
        Supported Platforms
        -------------------
        
        - Linux, FreeBSD, Mac OS X
        - Python 2.6, 2.7
        
        **Windows**: We acknowledge that Windows exists and make reasonable efforts
                     to maintain compatibility. Unfortunately we cannot guarantee
                     support at this time.
        
        
        Changes
        -------
        
        
        spyda 0.0.2 (2013-11-19)
        ........................
        
        - Updated the README
        - Added build status
        - Added install instructions
        
        
        spyda 0.0.1 (2013-11-19)
        ........................
        
        - Initiali Public Release
        
Keywords: Python Spider Web Crawling and Extraction Tool and Library
Platform: POSIX
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 2.7
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
