docs/libIndex.html

<html>
<head>
  <title>The Index Library</title>
  <link type="text/css" rel="stylesheet" href="../menu.css" />
  <link type="text/css" rel="stylesheet" href="../content.css" />
  <style type="text/css">
    td {
    vertical-align: top;
    }
  </style>
</head>

<body>

<!--#include virtual="../menu.html.incl"-->

<div id="content">

<h1>The Index Library</h1>

  <p><b>Table of Contents</b></p>
  <ul>
    <li><a href="#philosophy">Design Philosophy</a></li>
    <li><a href="#classes">Classes</a>
      <ul>
        <li><a href="#entity">Entity</a></li>
        <li><a href="#astlocation">ASTLocation</a></li>
        <li><a href="#declreferencemap">DeclReferenceMap</a></li>
      </ul>
    </li>
    <li><a href="#functions">Functions</a>
      <ul>
        <li><a href="#resolveloc">ResolveLocationInAST</a></li>
      </ul>
    </li>
    <li><a href="#astfiles">AST Files</a></li>
    <li><a href="#indextest">index-test tool</a>
      <ul>
        <li><a href="#indextestusage">Usage</a></li>
        <li><a href="#indextestexamples">Examples</a></li>
      </ul>
    </li>
</ul>
    
<h2 id="philosophy">Design Philosophy</h2>
  
<p> The Index library is meant to provide the basic infrastructure for
 cross-translation-unit analysis and is primarily focused on indexing
  related functionality. It provides an API for clients that need to
 accurately map the AST nodes of the ASTContext to the locations in the source files.
It also allows them to analyze information across multiple translation units.</p>

<p>As a "general rule", ASTContexts are considered the primary source of
information that a client wants about a translation unit. There will be no such class as an
 "indexing database" that stores, for example, source locations of identifiers separately from ASTContext.
All the information that a client needs from a translation unit will be extracted from the ASTContext.</p>

<h2 id="classes">Classes</h2>

<h3 id="entity">Entity</h3>

<p>To be able to reason about semantically the same Decls that are contained in multiple ASTContexts, the 'Entity' class was introduced.
An Entity is an ASTContext-independent "token" that can be created from a Decl (and a typename in the future) with
the purpose to "resolve" it into a Decl belonging to another ASTContext. Some examples to make the concept of Entities more clear:</p>

<p>
t1.c:
<pre class="code_example">
void foo(void);
void bar(void);
</pre>
</p>

<p>
t2.c:
<pre class="code_example">
void foo(void) {
}
</pre>
</p>

<p>
Translation unit <code>t1.c</code> contains 2 Entities <code>foo</code> and <code>bar</code>, while <code>t2.c</code> contains 1 Entity <code>foo</code>.
Entities are uniqued in such a way that the Entity* pointer for <code>t1.c/foo</code> is the same as the Entity* pointer for <code>t2.c/foo</code>.
An Entity doesn't convey any information about the declaration, it is more like an opaque pointer used only to get the
associated Decl out of an ASTContext so that the actual information for the declaration can be accessed.
Another important aspect of Entities is that they can only be created/associated for declarations that are visible outside the
translation unit. This means that for:
</p>
<p>
t3.c:
<pre class="code_example">
static void foo(void);
</pre>
</p>
<p>
there can be no Entity (if you ask for the Entity* of the static function <code>foo</code> you'll get a null pointer).
This is for 2 reasons:
<ul>
<li>To preserve the invariant that the same Entity* pointers refer to the same semantic Decls.
   In the above example <code>t1.c/foo</code> and <code>t2.c/foo</code> are the same, while <code>t3.c/foo</code> is different.</li>
<li>The purpose of Entity is to get the same semantic Decl from multiple ASTContexts. For a Decl that is not visible
   outside of its own translation unit, you don't need an Entity since it won't appear in another ASTContext.</li>
</ul>   
</p>
   
<h3 id="astlocation">ASTLocation</h3>

Encapsulates a "point" in the AST tree of the ASTContext.
It represents either a Decl*, or a Stmt* along with its immediate Decl* parent.
An example for its usage is that libIndex will provide the references of <code>foo</code> in the form of ASTLocations,
"pointing" at the expressions that reference <code>foo</code>.

<h3 id="declreferencemap">DeclReferenceMap</h3>

Accepts an ASTContext and creates a mapping from NamedDecls to the ASTLocations that reference them (in the same ASTContext).

<h2 id="functions">Functions</h2>

<h3 id="resolveloc">ResolveLocationInAST</h3>

A function that accepts an ASTContext and a SourceLocation which it resolves into an ASTLocation.

<h2 id="astfiles">AST Files</h2>

The precompiled headers implementation of clang (<a href="http://clang.llvm.org/docs/PCHInternals.html">PCH</a>) is ideal for storing an ASTContext in a compact form that
will be loaded later for AST analysis. An "AST file" refers to a translation unit that was "compiled" into a precompiled header file.

<h2 id="indextest">index-test tool</h2>

<h3 id="indextestusage">Usage</h3>

A command-line tool that exercises the libIndex API, useful for testing its features.
As input it accepts multiple AST files (representing multiple translation units) and a few options:

<p>
<pre class="code_example">
   -point-at  [file:line:column]
</pre>
Resolves a [file:line:column] triplet into a ASTLocation from the first AST file. If no other option is specified, it prints the ASTLocation.
It also prints a declaration's associated doxygen comment, if one is available.
</p>

<p>
<pre class="code_example">
   -print-refs
</pre>
Prints the ASTLocations that reference the declaration that was resolved out of the [file:line:column] triplet
</p>

<p>
<pre class="code_example">
   -print-defs
</pre>
Prints the ASTLocations that define the resolved declaration
</p>

<p>
<pre class="code_example">
   -print-decls
</pre>
Prints the ASTLocations that declare the resolved declaration
</p>

<h3 id="indextestexamples">Examples</h3>

<p>
Here's an example of using index-test:
</p>

<p>
We have 3 files,
</p>

<p>
foo.h:
<pre class="code_example">
extern int global_var;

void foo_func(int param1);
void bar_func(void);
</pre>

t1.c:
<pre class="code_example">
#include "foo.h"

void foo_func(int param1) {
  int local_var = global_var;
  for (int for_var = 100; for_var < 500; ++for_var) {
    local_var = param1 + for_var;
  }
  bar_func();
}
</pre>

t2.c:
<pre class="code_example">
#include "foo.h"

int global_var = 10;

void bar_func(void) {
  global_var += 100;
  foo_func(global_var);
}
</pre>
</p>

<p>
You first get AST files out of <code>t1.c</code> and <code>t2.c</code>:

<pre class="code_example">
$ clang -emit-ast t1.c -o t1.ast
$ clang -emit-ast t2.c -o t2.ast
</pre>
</p>

<p>
Find the ASTLocation under this position of <code>t1.c</code>:
<pre class="code_example">
[...]
void foo_func(int param1) {
  int local_var = global_var;
                      ^
[...]
</pre>

<pre class="code_example">
$ index-test t1.ast -point-at t1.c:4:23
> [Decl: Var local_var | Stmt: DeclRefExpr global_var] &lt;t1.c:4:19, t1.c:4:19>
</pre>
</p>

<p>
Find the declaration:

<pre class="code_example">
$ index-test t1.ast -point-at t1.c:4:23 -print-decls
> [Decl: Var global_var] &lt;foo.h:1:12, foo.h:1:12>
</pre>
</p>

<p>
Find the references:

<pre class="code_example">
$ index-test t1.ast t2.ast -point-at t1.c:4:23 -print-refs
> [Decl: Var local_var | Stmt: DeclRefExpr global_var] &lt;t1.c:4:19, t1.c:4:19>
> [Decl: Function bar_func | Stmt: DeclRefExpr global_var] &lt;t2.c:6:3, t2.c:6:3>
> [Decl: Function bar_func | Stmt: DeclRefExpr global_var] &lt;t2.c:7:12, t2.c:7:12>
</pre>
</p>

<p>
Find definitions:

<pre class="code_example">
$ index-test t1.ast t2.ast -point-at t1.c:4:23 -print-defs
> [Decl: Var global_var] &lt;t2.c:3:5, t2.c:3:18>
</pre>
</p>

</div>

</body>
</html>