"""Subpackage for indexing data.

Note that, while there is documentation for all the code in this
package, these modules/classes are considered internal and not
intended for use outside of the awsshell.  It's very likely that
these interfaces will change over time and may have backwards incompatible
changes as improvements are made.

This package contains modules for working with the index data that the AWS
Shell uses.  It manages the creation and loading of the AWS Shell index files.

In this context, an index file is specifically used to speed up various
operations performed by the AWS CLI: autocompletion, pulling up docs,
server side resource completion, etc.  In typically consists of data constructed
in such a way to make it quick and easy to answer questions such as
"what are the available subcommands?  what are the docs for this operation?
what arguments does this command take?".

The AWS Shell supports has several types of index files:

* The local command completion index.
* The documentation index.
* The server side resource completion index.

They're split into separate files because they're used in different scenarios,
and generated by different means (command completion is driven by botocore
JSON models, and server side completion is driven by boto3 resource models).

Couple of things to know about the index files:

* The command completion and doc index are indexed by CLI version.  Each version
of the CLI can potentially add new commands/services, so we want to ensure that
we have an index per version so we can offer the correct autocompletion and
documentation based on what CLI version you have installed.
* Command completion and documentation are automatically generated based
on demand. If we notice you don't have an index available for the particular
CLI version, we automatically generate the data.
* The total amount of documentation for every command for every service is
large.  We want to avoid loading the entire docset into memory, so we're not
using JSON.  Instead we're using a DBM interface (via ``shelve``).  May
also consider sqlite.
* Server side resource completion is *not* generated on demand.  This package
contains code for generating the a "reverse index" derived from boto3 docs.
This is primarily done while we explore to what extent the boto3 resource
models have the information we need.  To be fair, the resource models were
never intended to be used in the mechanism in which we're using them, so it's
very possible we may need to enhance the models by hand.  This is why we
pre-generate the index for server side completion up front so we can add hand
modifications if needed.  It's very possible that we may be able to come up
with something sufficiently sophisticated enough where we can change this index
to by autogenerated on demand like the other two.
* While the indexes are grouped in a single subpackage for discoverability,
they (purposefully) do not share the same interface for usage.  Each
index needs to answer different questions, so it does not make sense
for them to share the same interface.

Usage
=====

Loading the completion index

.. code-block:: python

  # Default usage:

  loader = CompletionIndex()
  # Load based on the current CLI version you have installed
  # (i.e awscli.__version__)
  index = loader.load_index()

  # Load from a specific directory.
  loader = CompletionIndex(cache_dir='/tmp/mycache')
  index = loader.load_index()

  # Load a specific version.

  loader = CompletionIndex()
  index = loader.load_index(version='1.9.1')

  # Generating an index.

  loader = CompletionIndex()
  # You don't give it a version because we can only
  # generate an index based on the version we can import.
  completion_index = loader.generate_index()
  # You don't need to give a filename, it will write it out
  # based on the current version.
  loader.write_index_to_file(completion_index)

  # Generate and write the index in one step.  It will also
  # return the generated index in case you need it.
  loader.generate_and_write_index()

  # This is some autocompleter could use the index loader:

  loader = CompletionIndex()
  if not loader.index_exists():
      index = loader.generate_and_write_index()
  else:
      index = loader.load_index()

  autocompleter = MyAutoCompleter(index)

"""
