WIP: LightGraphs Abstraction (#541)

* benchmarks

* add edgetype benchmark

katz centrality is broken.

* simplegraphs abstraction

* Edge is no longer a Pair

* pkgbenchmarks

* f

* remove data files from benchmarks

* simplegraphs, take 2

* more changes

* reshuffle

* fix tests

* more tests

* abstractions

* more tests

* tests and fixes

* trait fixes and tests - unrolling

* persistence and floyd-warshall

* make(di)graphs, through spanningtrees

* moved cliques, testing through connectivity.jl

* @jpfairbanks first round of review

* another fix

* all tests

* new simpletraits

* first cut at 0.6 compat

* squash

* update randgraphs.jl to use Channels over Tasks

Fixes deprecation warnings introduced in:

  JuliaLang/julia#19841

Changes an API interface:

  -function Graph(nvg::Int, neg::Int, edgestream::Task)
  +function Graph(nvg::Int, neg::Int, edgestream::Channel)

Iteration over Tasks is deprecated so now we iterate over the Channel.

* got rid of tasks in randgraphs

* graph -> g

* Add tutorials to section on docs (#547)

* Update README.md

* Update README.md

Made tutorials separate line and consistent with the other lines.

* type -> mutable struct

* more type -> mutable struct, plus OF detection for add_vertex!

* foo{T}(x::T) -> foo(x::T) where T

* test negative cycles

* test coverage

* manual cherry-pick of #551

* simplegraph/ -> simplegraphs, optimization for is_connected, some type simplifications

* re-add b-f tests

* Inferred (#554)

* core

* @inferred wherever possible

* empty -> zero

* test grid periodic=true

* oops

* redo graphmatrices tests

* linalg test fix

* loosen type restrictions in randgraphs functions

* readall -> readstring, and comment rationalization in randgraphs

* Fixes #555: graphmatrices convert incorrect on CA (#560)

CombinatorialAdjacency(CombinatorialAdjacency(g)) was returning the backing storage.
Fix includes tests.

* fixes #564

* one more test

* removed nv() and vertices() (#565)

* simpleedge tests

* test coverage

* short circuit B-F negative cycles, plus tests

* more test coverage

* more test coverage

* Docs (#567)

* docs, plus some various fixes (see blocking_flow).

* nodes -> vertices, node -> vertex, and more doc consistency

* doc fixes

* 1.0 -> 0.8

* docfix and benchmarks

* doc fixes

Author: sbromberger

.travis.yml

@@ -5,8 +5,8 @@ os:
 #  - osx
 
 julia:
-  - 0.5
-#  - nightly
+  # - 0.5
+ - nightly
 
 notifications:
   email: false

CONTRIBUTING.md

@@ -40,7 +40,7 @@ function f(g::AbstractGraph, v::Integer)
     return inner!(storage, g, v)
 end
 
-function inner!(storage::AbstractArray{Int,1}, g::AbstractGraph, v::Integer)
+function inner!(storage::AbstractVector{Int}, g::AbstractGraph, v::Integer)
     # some code operating on storage, g, and v.
     for i in 1:nv(g)
         storage[i] = v-i

README.md

@@ -28,6 +28,7 @@ better-optimized mechanisms.
 
 Additional functionality may be found in the companion package [LightGraphsExtras.jl](https://github.com/JuliaGraphs/LightGraphsExtras.jl).
 
+
 ## Documentation
 Full documentation is available at [GitHub Pages](https://juliagraphs.github.io/LightGraphs.jl/latest).
 Documentation for methods is also available via the Julia REPL help system.
@@ -38,91 +39,26 @@ A graph *G* is described by a set of vertices *V* and edges *E*:
 *G = {V, E}*. *V* is an integer range `1:n`; *E* is represented as forward
 (and, for directed graphs, backward) adjacency lists indexed by vertices. Edges
 may also be accessed via an iterator that yields `Edge` types containing
-`(src::Int, dst::Int)` values.
+`(src<:Integer, dst<:Integer)` values. Both vertices and edges may be integers
+of any type, and the smallest type that fits the data is recommended in order
+to save memory.
 
 *LightGraphs.jl* provides two graph types: `Graph` is an undirected graph, and
 `DiGraph` is its directed counterpart.
 
 Graphs are created using `Graph()` or `DiGraph()`; there are several options
-(see below for examples).
-
-Edges are added to a graph using `add_edge!(g, e)`. Instead of an edge type
-integers may be passed denoting the source and destination vertices (e.g.,
-`add_edge!(g, 1, 2)`).
+(see the tutorials for examples).
 
 Multiple edges between two given vertices are not allowed: an attempt to
 add an edge that already exists in a graph will result in a silent failure.
 
-Edges may be removed using `rem_edge!(g, e)`. Alternately, integers may be passed
-denoting the source and destination vertices (e.g., `rem_edge!(g, 1, 2)`). Note
-that, particularly for very large graphs, edge removal is a (relatively)
-expensive operation. An attempt to remove an edge that does not exist in the graph will result in an
-error.
-
-Use `nv(g)` and `ne(g)` to compute the number of vertices and edges respectively.
-
-`rem_vertex!(g, v)` alters the vertex identifiers. In particular, calling `n=nv(g)`, it swaps `v` and `n` and then removes `n`.
-
-`edges(g)` returns an iterator to the edge set. Use `collect(edge(set))` to fill
-an array with all edges in the graph.
 
 ## Installation
 Installation is straightforward:
 ```julia
 julia> Pkg.add("LightGraphs")
 ```
 
-## Usage Examples
-(all examples apply equally to `DiGraph` unless otherwise noted):
-
-```julia
-# create an empty undirected graph
-g = Graph()
-
-# create a 10-node undirected graph with no edges
-g = Graph(10)
-@assert nv(g) == 10
-
-# create a 10-node undirected graph with 30 randomly-selected edges
-g = Graph(10,30)
-
-# add an edge between vertices 4 and 5
-add_edge!(g, 4, 5)
-
-# remove an edge between vertices 9 and 10
-rem_edge!(g, 9, 10)
-
-# create vertex 11
-add_vertex!(g)
-
-# remove vertex 2
-# attention: this changes the id of vertex nv(g) to 2
-rem_vertex!(g, 2)
-
-# get the neighbors of vertex 4
-neighbors(g, 4)
-
-# iterate over the edges
-m = 0
-for e in edges(g)
-    m += 1
-end
-@assert m == ne(g)
-
-# show distances between vertex 4 and all other vertices
-dijkstra_shortest_paths(g, 4).dists
-
-# as above, but with non-default edge distances
-distmx = zeros(10,10)
-distmx[4,5] = 2.5
-distmx[5,4] = 2.5
-dijkstra_shortest_paths(g, 4, distmx).dists
-
-# graph I/O
-g = loadgraph("mygraph.jgz", :lg)
-savegraph("mygraph.gml", g, :gml)
-```
-
 ## Current functionality
 - **core functions:** vertices and edges addition and removal, degree (in/out/histogram), neighbors (in/out/all/common)
 
@@ -155,44 +91,50 @@ symmetric difference, blkdiag, induced subgraphs, products (cartesian/scalar)
 
 - **community:** modularity, community detection, core-periphery, clustering coefficients
 
-- **persistence formats:** proprietary compressed, [GraphML](http://en.wikipedia.org/wiki/GraphML), [GML](https://en.wikipedia.org/wiki/Graph_Modelling_Language), [Gexf](http://gexf.net/format), [DOT](https://en.wikipedia.org/wiki/DOT_(graph_description_language)), [Pajek NET](http://gephi.org/users/supported-graph-formats/pajek-net-format/)
+- **persistence formats:** proprietary compressed, [GraphML](http://en.wikipedia.org/wiki/GraphML), [GML](https://en.wikipedia.org/wiki/Graph_Modelling_Language), [Gexf](http://gexf.net/format), [DOT](https://en.wikipedia.org/wiki/DOT_(graph_description_language)), [Pajek NET](http://gephi.org/users/supported-graph-formats/pajek-net-format/), [Graph6](http://users.cecs.anu.edu.au/~bdm/data/formats.html)
 
-- **visualization:** integration with [GraphLayout](https://github.com/IainNZ/GraphLayout.jl), [TikzGraphs](https://github.com/sisl/TikzGraphs.jl), [GraphPlot](https://github.com/JuliaGraphs/GraphPlot.jl), [NetworkViz](https://github.com/abhijithanilkumar/NetworkViz.jl/)
+- **visualization:** integration with
+[GraphPlot](https://github.com/JuliaGraphs/GraphPlot.jl),
+[Plots](https://github.com/JuliaPlots/Plots.jl) via [PlotRecipes](https://github.com/JuliaPlots/PlotRecipes.jl), [GraphLayout](https://github.com/IainNZ/GraphLayout.jl), [TikzGraphs](https://github.com/sisl/TikzGraphs.jl),  [NetworkViz](https://github.com/abhijithanilkumar/NetworkViz.jl/)
 
 
 ## Core API
-These functions are defined as the public contract of the LightGraphs.AbstractGraph interface.
+These functions are defined as the public contract of the `LightGraphs.AbstractGraph` interface.
 
 ### Constructing and modifying the graph
-
-- add_edge!
-- rem_edge!
-- add_vertex!
-- add_vertices!
-- rem_vertex!
+- `Graph`
+- `DiGraph`
+- `add_edge!`
+- `rem_edge!`
+- `add_vertex!`, `add_vertices!`
+- `rem_vertex!`
+- `zero`
 
 ### Edge/Arc interface
-- src
-- dst
+- `src`
+- `dst`
+- `reverse`
+- `==`
+- Pair / Tuple conversion
 
 ### Accessing state
-- nv::Int
-- ne::Int
-- vertices (Iterable)
-- edges (Iterable)
-- neighbors
-- in_edges
-- out_edges
-- has_vertex
-- has_edge
-- has_self_loops (though this might be a trait or an abstract graph type)
+- `nv`
+- `ne`
+- `vertices` (Iterable)
+- `edges` (Iterable)
+- `neighbors`, `in_neighbors`, `out_neighbors`
+- `in_edges`
+- `out_edges`
+- `has_vertex`
+- `has_edge`
+- `has_self_loops` (though this might be a trait or an abstract graph type)
 
 
 ### Non-Core APIs
 These functions can be constructed from the Core API functions but can be given specialized implementations in order to improve performance.
 
-- adjacency_matrix
-- degree
+- `adjacency_matrix`
+- `degree`
 
 This can be computed from neighbors by default `degree(g,v) = length(neighbors(g,v))` so you don't need to implement this unless your type can compute degree faster than this method.
 
@@ -202,7 +144,8 @@ This can be computed from neighbors by default `degree(g,v) = length(neighbors(g
 * Julia 0.3: LightGraphs v0.3.7 is the last version guaranteed to work with Julia 0.3.
 * Julia 0.4: LightGraphs versions in the 0.6 series are designed to work with Julia 0.4.
 * Julia 0.5: LightGraphs versions in the 0.7 series are designed to work with Julia 0.5.
-* Julia 0.6: Some functionality might not work with prerelease / unstable / nightly versions of Julia. If you run into a problem on 0.6, please file an issue.
+* Julia 0.6: LightGraphs versions in the 0.8 series are designed to work with Julia 0.6.
+* Later versions: Some functionality might not work with prerelease / unstable / nightly versions of Julia. If you run into a problem, please file an issue.
 
 # Contributing and Reporting Bugs
 We welcome contributions and bug reports! Please see [CONTRIBUTING.md](https://github.com/JuliaGraphs/LightGraphs.jl/blob/master/CONTRIBUTING.md)

REQUIRE

@@ -1,8 +1,9 @@
-julia 0.5
+julia 0.6-
 GZip 0.2.20
 EzXML 0.3.0
 ParserCombinator 1.7.11
 JLD 0.6.3
 Distributions 0.10.2
 StatsBase 0.9.0
 DataStructures 0.5.0
+SimpleTraits 0.4.0

benchmark/edges.jl

@@ -1,5 +1,6 @@
 import Base: convert
-typealias P Pair{Int, Int}
+
+const P = Pair{Int, Int}
 
 convert(::Type{Tuple}, e::Pair) = (e.first, e.second)
 

benchmark/max-flow.jl

@@ -4,7 +4,7 @@
     p = 8.0 / n
     A = sprand(n,n,p)
     g = DiGraph(A)
-    cap = round(A*100)
+    cap = round.(A*100)
     @bench "n = $n" LightGraphs.maximum_flow($g, 1, $n, $cap)
   end
 end # max-flow

benchmarks/core.jl

@@ -0,0 +1,41 @@
+bg = BenchmarkGroup()
+SUITE["core"] = bg
+
+function bench_iteredges(g::AbstractGraph)
+  i = 0
+  for e in edges(g)
+    i += 1
+  end
+  return i
+end
+
+function bench_has_edge(g::AbstractGraph)
+    srand(1)
+    nvg = nv(g)
+    srcs = rand([1:nvg;], cld(nvg, 4))
+    dsts = rand([1:nvg;], cld(nvg, 4))
+    i = 0
+    for (s, d) in zip(srcs, dsts)
+        if has_edge(g, s, d)
+            i += 1
+        end
+    end
+    return i
+end
+
+
+EDGEFNS = [
+    bench_iteredges,
+    bench_has_edge
+]
+
+for fun in EDGEFNS
+    for (name, g) in GRAPHS
+        bg["edges","$fun","graph","$name"] = @benchmarkable $fun($g)
+    end
+
+    for (name, g) in DIGRAPHS
+        bg["edges","$fun","digraph","$name"] = @benchmarkable $fun($g)
+    end
+
+end

src/LightGraphs.jl

@@ -7,21 +7,25 @@ using DataStructures
 using EzXML
 using ParserCombinator: Parsers.DOT, Parsers.GML
 using StatsBase: fit, Histogram
+using SimpleTraits
 
 import Base: write, ==, <, *, ≈, convert, isless, issubset, union, intersect,
-            reverse, reverse!, blkdiag, getindex, setindex!, show, print, copy, in,
-            sum, size, sparse, eltype, length, ndims, transpose,
-            ctranspose, join, start, next, done, eltype, get, issymmetric, A_mul_B!
-
+            reverse, reverse!, blkdiag, isassigned, getindex, setindex!, show,
+            print, copy, in, sum, size, sparse, eltype, length, ndims, transpose,
+            ctranspose, join, start, next, done, eltype, get, issymmetric, A_mul_B!,
+            Pair, Tuple, zero
+export
+# Interface
+AbstractGraph, AbstractDiGraph, AbstractEdge, AbstractEdgeInter,
+Edge, Graph, DiGraph, vertices, edges, edgetype, nv, ne, src, dst,
+is_directed, add_vertex!, add_edge!, rem_vertex!, rem_edge!,
+has_vertex, has_edge, in_neighbors, out_neighbors,
 
 # core
-export AbstractGraph, Edge, Graph, DiGraph, vertices, edges, src, dst,
-fadj, badj, in_edges, out_edges, has_vertex, has_edge, is_directed,
-nv, ne, add_edge!, rem_edge!, add_vertex!, add_vertices!,
-indegree, outdegree, degree, degree_histogram, density, Δ, δ,
-Δout, Δin, δout, δin, neighbors, in_neighbors, out_neighbors,
-common_neighbors, all_neighbors, has_self_loops, num_self_loops,
-rem_vertex!,
+is_ordered, add_vertices!, indegree, outdegree, degree,
+Δout, Δin, δout, δin, Δ, δ, degree_histogram,
+neighbors, all_neighbors, common_neighbors,
+has_self_loops, num_self_loops, density, squash,
 
 # distance
 eccentricity, diameter, periphery, radius, center,
@@ -113,7 +117,10 @@ kruskal_mst, prim_mst,
 #biconnectivity and articulation points
 articulation, biconnected_components
 
-"""An optimized graphs package.
+"""
+    LightGraphs
+
+An optimized graphs package.
 
 Simple graphs (not multi- or hypergraphs) are represented in a memory- and
 time-efficient manner with adjacency lists and edge sets. Both directed and
@@ -127,14 +134,21 @@ explicit design decision that any data not required for graph manipulation
 (attributes and other information, for example) is expected to be stored
 outside of the graph structure itself. Such data lends itself to storage in
 more traditional and better-optimized mechanisms.
+
+[Full documentation](http://codecov.io/github/JuliaGraphs/LightGraphs.jl) is available,
+and tutorials are available at the
+[JuliaGraphsTutorials repository](https://github.com/JuliaGraphs/JuliaGraphsTutorials).
 """
 LightGraphs
-
+include("interface.jl")
+include("deprecations.jl")
 include("core.jl")
-    include("digraph.jl")
+    include("graphtypes/simplegraphs/SimpleGraphs.jl")
+    const Graph = SimpleGraphs.SimpleGraph
+    const DiGraph = SimpleGraphs.SimpleDiGraph
+    const Edge = SimpleGraphs.SimpleEdge
+
     include("digraph-transitivity.jl")
-    include("graph.jl")
-        include("edgeiter.jl")
         include("traversals/graphvisit.jl")
             include("traversals/bfs.jl")
             include("traversals/dfs.jl")