
""""
=================
Statistics Module
=================

We generate statistics for different files in a repository. Since a repository can contain source
files written in different programming languages, we use different map-reduce classes to
process these. The returned objects can provide both language-dependent and
language-independent statistics, which can then be further aggregated or grouped.

The statistics module is organized in the following way:

* *languages/* -> Language-specific map-reducers (e.g. Python, Ruby etc.)
* *aggregators/* -> Map-reducers for aggregating data generated by the language-specific map-reducers

Example:

*languages/python/functions.py* takes as input a file node object and returns summary statistics of
all functions defined in that file (e.g. function length, name length). *languages/ruby/functions.py*
does the same for Ruby files. *languages/base* contains base classes that can be used as a basis
and tweaked for different languages.

tests.py -> {
    'filename' : '/testsuite/python/tests.py',
    'language' : 'python',
    'functions' : {
        'n' : 421,
        'average_length' = 12.2345112,
        'length_distribution' : {...},
    },
    'classes' : {
        ...
    }
}

tests.rb -> {
    'filename' : '/testsuite/ruby/tests.rb',
    'language' : 'ruby',
    ...
}

FunctionAggregator:
    supported_languages = ['python','ruby','...']

    Mapper:
        Emits a list of key/value pairs with function statistics:
        [
            ['python','[root]','/testsuite','/testsuite/python']
                :
                {
                'n' : 421,
                'average_length' : 12.23...
                ...
                },
            ['ruby','[root]','/testsuite','/testsuite/ruby']
                :{
                'n' : ...
                }
        ]
    Reducer:
        Performs aggregation over the given keys, returns a list of summary statistics:
            {
                'python' : {...python summary statistics...}
                '[root]' : {...inter-language statistics over whole repository...},
                '/testsuite' : {...inter-language statistics over /testsuite directory...}
            }

"""