make gitdb tests not require being ran from within gitdb git repository

[yarikoptic]

Although I also got confused with #7 suggesting that gitdb is "dead" and absorbed into GitPython but it seems to be live and kicking out new releases ;) So I will assume that.

It would be nice if gitdb tests could be ran on a source distribution of gitdb, i.e. without relying on having .git. Now even if I initialize some dummy repository

mkdir TESTGITDB
cd TESTGITDB
git init; for c in 1 2 3; do echo $c >| 1; git add 1; git commit -m "commit $c"; done

manually and point to it as directed by messages I get:

> GITDB_TEST_GIT_REPO_BASE=$PWD/TESTGITDB/.git nosetests -s -v gitdb/test
...
======================================================================
ERROR: test_reading (gitdb.test.db.test_git.TestGitDB)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/db/test_git.py", line 22, in test_reading
    assert 1 < len(gdb.databases()) < 4
  File "/home/yoh/deb/gits/python-gitdb/gitdb/db/base.py", line 223, in databases
    return tuple(self._dbs)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/util.py", line 237, in __getattr__
    self._set_cache_(attr)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/db/git.py", line 58, in _set_cache_
    raise InvalidDBRoot(self.root_path())
InvalidDBRoot: /home/yoh/deb/gits/python-gitdb/gitdb/test/fixtures/../../../.git/objects

======================================================================
ERROR: test_pack_random_access (gitdb.test.performance.test_pack.TestPackedDBPerformance)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/lib.py", line 45, in wrapper
    return func(self, *args, **kwargs)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/performance/test_pack.py", line 54, in test_pack_random_access
    (ns, len(pdb.entities()), elapsed, ns / elapsed), file=sys.stderr)
ZeroDivisionError: float division by zero

======================================================================
ERROR: test_pack_writing (gitdb.test.performance.test_pack_streaming.TestPackStreamingPerformance)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/lib.py", line 45, in wrapper
    return func(self, *args, **kwargs)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/performance/test_pack_streaming.py", line 58, in test_pack_writing
    PackEntity.write_pack((pdb.stream(sha) for sha in pdb.sha_iter()), ostream.write, object_count=ni)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/pack.py", line 979, in write_pack
    "Expected to write %i objects into pack, but received only %i from iterators" % (object_count, actual_count))
ValueError: Expected to write 1000 objects into pack, but received only 0 from iterators

======================================================================
ERROR: test_base (gitdb.test.test_example.TestExamples)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/test_example.py", line 42, in test_base
    ldb.store(istream)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/db/loose.py", line 184, in store
    fd, tmp_path = tempfile.mkstemp(prefix='obj', dir=self._root_path)
  File "/usr/lib/python2.7/tempfile.py", line 308, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags)
  File "/usr/lib/python2.7/tempfile.py", line 239, in _mkstemp_inner
    fd = _os.open(file, flags, 0600)
OSError: [Errno 2] No such file or directory: '/home/yoh/deb/gits/python-gitdb/gitdb/test/fixtures/../../../.git/objects/objZdoPUA'

======================================================================
FAIL: test_writing (gitdb.test.db.test_ref.TestReferenceDB)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/lib.py", line 60, in wrapper
    return func(self, path)
  File "/home/yoh/deb/gits/python-gitdb/gitdb/test/db/test_ref.py", line 46, in test_writing
    assert len(rdb.databases()) == 1
AssertionError

----------------------------------------------------------------------
Ran 24 tests in 13.322s

FAILED (SKIP=1, errors=4, failures=1)

I think it would have been best if there was a fixture for the tests battery which would initiate such a repository using git present on the system (thus testing against whatever git available) and do not even rely on me doing above dance manually. If some tests are really dependent on gitdb/.git -- I guess they could Skip if no upstairs .git available.

Thanks in advance!

[Byron]

Maybe I am misunderstanding this, but in any case, I am sure we can figure this out !
First of all, gitdb is alive and kicking indeed. There was the plan to merge it into the 'experiment-2012' branch, but this one is now obsolete and deprecated.

What I find odd is that the gitdb tests don't work for you out of the box, as you don't have to specify a custom repository anymore at all. If the environment variable is unset, it will just use it's own 'gitdb' repository which works fine. Travis for example does it that way.

If you want to use that variable, than it expects to be pointing to some moderately large repository that has at least one pack. I used the git source repository for this at some point.

For me, the system seems to be working the way it is, and I'd rather remove this environment variable than try to include some massive fixture (if that is what you suggested).
Please let me know what you think.

[yarikoptic]

"it will just use it's own 'gitdb' repository which works fine" -- that is what I was trying to avoid as well. E.g. for debian package, we have only sources, no gitdb's .git is shipped along. But I still would like to run tests to verify that gitdb operates correctly. I hope it clears out my intentions? ;)

[Byron]

Alright, then I'd recommend cloning the gitdb repo as part of your build scripts, and point the environment variable there. This seems like the most bullet-proof way of doing it, as this is how I run my tests too.
Does that sound applicable to you ?

[yarikoptic]

wouldn't work for me either since network connectivity is not guaranteed! ;-) (that is a part of Debian policy carved in stone)

[Byron]

oh ... well. The point is that the repository it uses needs to be sufficiently large, and must have packs. Maybe what you could do is to just keep a clone of gitdb for testing as part of your build-script. Doesn't have to be the latest one, as far as I am concerned. Something like that must be possible.
Does that make sense, it is feasible ?

[yarikoptic]

well... I would be lying if I say that it is not feasible... there is even a 3.0 (git) format for source packages in Debian so the sources could be provided with .git and I could then drag entire history of gitdb and debian packaging within. may be at some point I will actually do that but not now -- package is maintained under SVN of the python-modules team in Debian -- I would need to migrate it away etc. But meanwhile (I am not even original maintainer for gitdb/gitpython, just picked up to provide fresh releases for others and my own good) I think, if you still allow testing against some other repository, I would instantiate it from scratch and just skip (via a patch or just argument to nose) the tests requiring access to packs or anything else gitdb/large git-repo specific. ATM tests are not excercised at all at package build time -- so some tests already will be better than no tests at all.

[yarikoptic]

do you see any tests among above error'ing/failing which could be fixed up counting on lean test git repo existing?

[Byron]

Yeah, I see your point. The first one for instance actively tries to access it's own repository. Tests should at least be fixed to use the one provided by the environment variable at all times.
Then it would just be up to you to setup a repository which is big enough.
Will work on that right away, thanks for guiding me to this realisation.
To be honest, I didn't even know these tests anymore and just assumed they all use the environment variable.

[yarikoptic]

Thank you very much in advance!

[Byron]

Alright, it should work now. Please let me know if you find anything else though.