:orphan:



.. _sphx_glr_auto_examples:

Documentation
=============

We welcome contributions to our documentation via GitHub pull requests, whether it's fixing a typo or authoring an entirely new tutorial or guide.
If you're thinking about contributing documentation, please see :ref:`sphx_glr_auto_examples_howtos_run_doc.py`.


.. raw:: html

    <div class="sphx-glr-clear"></div>



.. _sphx_glr_auto_examples_core:

Core Tutorials: New Users Start Here!
-------------------------------------

If you're new to gensim, we recommend going through all core tutorials in order.
Understanding this functionality is vital for using gensim effectively.



.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial introduces Documents, Corpora, Vectors and Models: the basic concepts and terms n...">

.. only:: html

 .. figure:: /auto_examples/core/images/thumb/sphx_glr_run_core_concepts_thumb.png
     :alt: Core Concepts

     :ref:`sphx_glr_auto_examples_core_run_core_concepts.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/core/run_core_concepts

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates transforming text into a vector space representation.">

.. only:: html

 .. figure:: /auto_examples/core/images/thumb/sphx_glr_run_corpora_and_vector_spaces_thumb.png
     :alt: Corpora and Vector Spaces

     :ref:`sphx_glr_auto_examples_core_run_corpora_and_vector_spaces.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/core/run_corpora_and_vector_spaces

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces transformations and demonstrates their use on a toy corpus. ">

.. only:: html

 .. figure:: /auto_examples/core/images/thumb/sphx_glr_run_topics_and_transformations_thumb.png
     :alt: Topics and Transformations

     :ref:`sphx_glr_auto_examples_core_run_topics_and_transformations.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/core/run_topics_and_transformations

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates querying a corpus for similar documents. ">

.. only:: html

 .. figure:: /auto_examples/core/images/thumb/sphx_glr_run_similarity_queries_thumb.png
     :alt: Similarity Queries

     :ref:`sphx_glr_auto_examples_core_run_similarity_queries.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/core/run_similarity_queries
.. raw:: html

    <div class="sphx-glr-clear"></div>



.. _sphx_glr_auto_examples_tutorials:

Tutorials: Learning Oriented Lessons
------------------------------------

Learning-oriented lessons that introduce a particular gensim feature, e.g. a model (Word2Vec, FastText) or technique (similarity queries or text summarization).



.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces Gensim&#x27;s Word2Vec model and demonstrates its use on the `Lee Evaluation Corpus &lt;http...">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_word2vec_thumb.png
     :alt: Word2Vec Model

     :ref:`sphx_glr_auto_examples_tutorials_run_word2vec.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_word2vec

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces Gensim&#x27;s Doc2Vec model and demonstrates its use on the `Lee Corpus &lt;https://hekyll.s...">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_doc2vec_lee_thumb.png
     :alt: Doc2Vec Model

     :ref:`sphx_glr_auto_examples_tutorials_run_doc2vec_lee.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_doc2vec_lee

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces Gensim&#x27;s fastText model and demonstrates its use on the Lee Corpus. ">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_fasttext_thumb.png
     :alt: FastText Model

     :ref:`sphx_glr_auto_examples_tutorials_run_fasttext.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_fasttext

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces the Annoy library for similarity queries on top of vectors learned by Word2Vec. ">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_annoy_thumb.png
     :alt: Fast Similarity Queries with Annoy and Word2Vec

     :ref:`sphx_glr_auto_examples_tutorials_run_annoy.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_annoy

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Introduces Gensim&#x27;s LDA model and demonstrates its use on the NIPS corpus.">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_lda_thumb.png
     :alt: LDA Model

     :ref:`sphx_glr_auto_examples_tutorials_run_lda.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_lda

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates using Gensim&#x27;s implemenation of the SCM.">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_scm_thumb.png
     :alt: Soft Cosine Measure

     :ref:`sphx_glr_auto_examples_tutorials_run_scm.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_scm

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates using Gensim&#x27;s implemenation of the WMD.">

.. only:: html

 .. figure:: /auto_examples/tutorials/images/thumb/sphx_glr_run_wmd_thumb.png
     :alt: Word Mover's Distance

     :ref:`sphx_glr_auto_examples_tutorials_run_wmd.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/tutorials/run_wmd
.. raw:: html

    <div class="sphx-glr-clear"></div>



.. _sphx_glr_auto_examples_howtos:

How-to Guides: Solve a Problem
------------------------------

These **goal-oriented guides** demonstrate how to **solve a specific problem** using gensim.



.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates simple and quick access to common corpora and pretrained models. ">

.. only:: html

 .. figure:: /auto_examples/howtos/images/thumb/sphx_glr_run_downloader_api_thumb.png
     :alt: How to download pre-trained models and corpora

     :ref:`sphx_glr_auto_examples_howtos_run_downloader_api.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/howtos/run_downloader_api

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="How to author documentation for Gensim. ">

.. only:: html

 .. figure:: /auto_examples/howtos/images/thumb/sphx_glr_run_doc_thumb.png
     :alt: How to Author Gensim Documentation

     :ref:`sphx_glr_auto_examples_howtos_run_doc.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/howtos/run_doc

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Shows how to reproduce results of the &quot;Distributed Representation of Sentences and Documents&quot; p...">

.. only:: html

 .. figure:: /auto_examples/howtos/images/thumb/sphx_glr_run_doc2vec_imdb_thumb.png
     :alt: How to reproduce the doc2vec 'Paragraph Vector' paper

     :ref:`sphx_glr_auto_examples_howtos_run_doc2vec_imdb.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/howtos/run_doc2vec_imdb

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Demonstrates how you can visualize and compare trained topic models.">

.. only:: html

 .. figure:: /auto_examples/howtos/images/thumb/sphx_glr_run_compare_lda_thumb.png
     :alt: How to Compare LDA Models

     :ref:`sphx_glr_auto_examples_howtos_run_compare_lda.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/howtos/run_compare_lda
.. raw:: html

    <div class="sphx-glr-clear"></div>



.. _sphx_glr_auto_examples_other:

Other Resources
---------------

Blog posts, tutorial videos, hackathons and other useful Gensim resources, from around the internet.

- *Use FastText or Word2Vec?* Comparison of embedding quality and performance. `Jupyter Notebook <https://github.com/RaRe-Technologies/gensim/blob/ba1ce894a5192fc493a865c535202695bb3c0424/docs/notebooks/Word2Vec_FastText_Comparison.ipynb>`__
- Multiword phrases extracted from *How I Met Your Mother*. `Blog post by Mark Needham <http://www.markhneedham.com/blog/2015/02/12/pythongensim-creating-bigrams-over-how-i-met-your-mother-transcripts/>`__
- *Using Gensim LDA for hierarchical document clustering*. `Jupyter notebook by Brandon Rose <http://brandonrose.org/clustering#Latent-Dirichlet-Allocation>`__
- *Evolution of Voldemort topic through the 7 Harry Potter books*. `Blog post <http://rare-technologies.com/understanding-and-coding-dynamic-topic-models/>`__
- *Movie plots by genre*: Document classification using various techniques: TF-IDF, word2vec averaging, Deep IR, Word Movers Distance and doc2vec. `Github repo <https://github.com/RaRe-Technologies/movie-plots-by-genre>`__
- *Word2vec: Faster than Google? Optimization lessons in Python*, talk by Radim Řehůřek at PyData Berlin 2014. `Youtube video <https://www.youtube.com/watch?v=vU4TlwZzTfU>`__
- *Word2vec & friends*, talk by Radim Řehůřek at MLMU.cz 7.1.2015. `Youtube video <https://www.youtube.com/watch?v=wTp3P2UnTfQ>`__

..
   - ? `Making an Impact with NLP <https://www.youtube.com/watch?v=oSSnDeOXTZQ>`__ -- Pycon 2016 Tutorial by Hobsons Lane
   - ? `NLP with NLTK and Gensim <https://www.youtube.com/watch?v=itKNpCPHq3I>`__ -- Pycon 2016 Tutorial by Tony Ojeda, Benjamin Bengfort, Laura Lorenz from District Data Labs
   - ? `Word Embeddings for Fun and Profit <https://www.youtube.com/watch?v=lfqW46u0UKc>`__ -- Talk at PyData London 2016 talk by Lev Konstantinovskiy. See accompanying `repo <https://github.com/RaRe-Technologies/movie-plots-by-genre>`__
   - ? English Wikipedia; TODO: convert to proper .py format
   - ? `Colouring words by topic in a document, print words in a
     topics <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/topic_methods.ipynb>`__
   - ? `Topic Coherence, a metric that correlates that human judgement on topic quality. <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/topic_coherence_tutorial.ipynb>`__
   - ? `America's Next Topic Model slides <https://speakerdeck.com/tmylk/americas-next-topic-model-at-pydata-berlin-august-2016?slide=7>`__
      - How to choose your next topic model, presented at Pydata Berlin 10 August 2016 by Lev Konstantinovsky
   - ?  `Dynamic Topic Modeling and Dynamic Influence Model Tutorial <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/dtm_example.ipynb>`__
   - ?  `Python Dynamic Topic Modelling Theory and Tutorial <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/ldaseqmodel.ipynb>`__
   - ? `Word Movers Distance for Yelp Reviews tutorial <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/WMD_tutorial.ipynb>`__
     - FIXME WMD superceded by soft cosine similarity = faster and better? any numbers / tutorials for that?
   - ? `Great illustration of corpus preparation <https://linanqiu.github.io/2015/10/07/word2vec-sentiment/>`__, `Code <https://github.com/linanqiu/word2vec-sentiments>`__
     - ? `Alternative <https://medium.com/@klintcho/doc2vec-tutorial-using-gensim-ab3ac03d3a1#.nv2lxvbj1>`__,
     - ? `Alternative 2 <https://districtdatalabs.silvrback.com/modern-methods-for-sentiment-analysis>`__
   - ? `Doc2Vec on customer reviews <http://multithreaded.stitchfix.com/blog/2015/03/11/word-is-worth-a-thousand-vectors/>`__
   - ? `Doc2Vec on Airline Tweets Sentiment Analysis <https://www.zybuluo.com/HaomingJiang/note/462804>`__
   - ? `Deep Inverse Regression with Yelp Reviews <https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/deepir.ipynb>`__ (Document Classification using Bayesian Inversion and several word2vec models, one for each class)


.. raw:: html

    <div class="sphx-glr-clear"></div>



.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-gallery


  .. container:: sphx-glr-download sphx-glr-download-python

    :download:`Download all examples in Python source code: auto_examples_python.zip <//Volumes/work/workspace/gensim/trunk/docs/src/auto_examples/auto_examples_python.zip>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

    :download:`Download all examples in Jupyter notebooks: auto_examples_jupyter.zip <//Volumes/work/workspace/gensim/trunk/docs/src/auto_examples/auto_examples_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
