Documentation and cleanning

howto_compile.md

+Librairies
+==========
+
+LMDB
+----
+
+This lib is necessary for all executables. Let your distribution do the job.
+
+osmpbf
+------
+
+This lib is necessary only for the creation of route database. Do not do this in
+the rv2 dir but in the dir which contains rv2 (or change rv2 src makefile).
+
+{{{
+    git clone https://github.com/inphos42/osmpbf.git
+    cd osmpbf
+    git clone https://github.com/inphos42/generics.git
+    cmake .
+    make
+}}}
+
+Build
+=====
+
+Go in the src dir, and use make for the target you want. The following target
+are available:
+
+ - elevation_create_database: creation of elevation database from srtm files
+ 
+ - elevation_query_database: query of elevation by coordinates
+ 
+ - lmdb_rewriter: rewrite a lmdb database, usefull to rewrite big sparse file as
+   little dense one.
+
+ - route_create_database_from_pbf: (needs osmpbf lib) creation of the route
+   database (constituted by two db, lookup and nodes) from a pbf file and using
+   elevation database.
+
+ - route_query_lookup_database: query lookup db in route db to found nodes id
+   near the given coordinates. Nodes are searched on the same connected
+   component.
+
+ - route_query_nodes_database: query nodes db in route db to have informations
+   about given nodes id.
+
+ - route: Give a route between given coordinated. The goal of the project.
+

howto_generate_database.md

+Download pbf file
+=================
+
+I sure you can do that yourself
+
+
+Build elevation database
+========================
+
+This database is not usefull for routing. This database is only usefull for
+building routing database. This database is not needed to be generated when the
+pbf is updated if the pdf cover the same area.
+
+Analyze pbf
+-----------
+
+{{{
+    ./src/analyze_pbf SOMEWHERE/france-latest.osm.pbf > SOMEWHERE/list_needed_srtm
+}}}
+
+The `analyze_pbf` program give:
+
+ - in stdout the list of needed srtm tiles.
+
+ - is stderr the number of couple (way,node)s usefull for routing. Let N the
+   number of couple (way,node)s returned by analyze_pbf. The prediction are the
+   following:
+
+   - (150 Bytes) × N : the routing database size
+
+   - (157 Bytes) × N : the minimal amount of RAM to create the database
+
+   - (253 Bytes) × N : a /good/ amount of RAM to create the database
+
+   - (421 Bytes) × N : the optimal amount of RAM to create the database (more
+           RAM is useless).
+
+   Keep these value in mind for next part.
+
+Downloading SRTM tiles
+----------------------
+
+With the `list_needed_srtm`, download the tiles. A script is provided in scripts.
+This script is the definition of brutality.
+
+
+{{{
+    ./scripts/dlsrtm SOMEWHERE/list_needed_srtm SOMEWHERE/srtmzip
+}}}
+
+
+Unzip SRTM files
+----------------
+
+{{{
+    mkdir -p SOMEWHERE/srtm
+    find SOMEWHERE/srtmzip/ -print0 | xargs -0 -n1 -I{} unzip {} -d SOMEWHERE/srtm
+}}}
+
+We don't need SOMEWHERE/srtmzip anymore. You can remove it.
+
+
+Creating elevation db
+---------------------
+
+{{{
+    ./src/elevation_create_database SOMEWHERE/ele_db_sparse SOMEWHERE/srtm/*.hgt
+}}}
+
+This elevation db is directly usable. Buto create this elevation db, the program
+use a writable map, and the elevation database is a big sparse file. If you are
+not planning to backup, or transfer this file, you can skip the next step, but
+the time used in the next step is very small, and this is not a good idea in
+general case.
+
+
+Converting elevation db
+-----------------------
+
+We use the agnostic lmdb_rewriter programm. It read source database, and
+re-write it sequencialy (without using writable map). The two database are
+equivalent, but the created elevation db from last step is a big sparse file,
+and after this step, the elevation db is a small dense file.
+
+{{{
+    ./src/lmdb_rewriter SOMEWHERE/ele_db_sparse SOMEWHERE/ele_db
+}}}
+    
+SOMEWHERE/ele_db_sparse is not needed anymore.
+
+
+Keep it
+-------
+
+Keep this elevation database. It is not needed to recreate it when the pbf is
+updated. Except the case when new area is added to the pbf which are not in the
+elevation database, but in this case, you will know it, the routing db (next
+part) creation will fail, with a explanation.
+
+
+Testing the elevation database
+------------------------------
+
+The program elevation_query_database use coupe (lon,lat) in stdin and give
+elevation in stdout. For example, to get the elevation of the point given by
+coords (lon=1.017, lat=48.513) use:
+
+{{{
+    echo 1.017 48.513 | ./src/elevation_query_database SOMEWHERE/ele_db
+}}}
+
+
+Building routing database
+=========================
+
+You need to rebuild it each time the pbf is updated (if you need a updated
+routing, of course). To create this database, you need to have:
+
+- the pbf (obviously)
+
+- a elevation database which cover the area
+
+- a large amount of RAM (see predictions in previous part obtained by pbf
+  analysis)
+
+{{{
+    ./src/route_create_databases_from_pbf SOMEWHERE/france-latest.osm.pbf SOMEWHERE/ele_db SOMEWHERE/route_db
+}}}
+
+Have a cup of tea.
+
+Testing routing database
+------------------------
+
+The routing database is made by two databases, the lookup database, and the node
+database.
+
+First we test the lookup database. For that the program
+route_query_lookup_database do the job. This program take a set of coupe
+(lon,lat) terminated by 0. For example, to get the nodes in the same connected
+component which are near points of coords:
+ - (lon=0.044, lat=45.661)
+ - (lon=0.169, lat=45.648)
+ - (lon=0.240, lat=45.737)
+
+use the following command:
+
+{{{
+    echo 0.044 45.661 0.169 45.648 0.240 45.737 0 \
+      | ./src/route_query_lookup_database SOMEWHERE/route_db
+}}}
+
+Second we test the node db. You must have node id (you can use nodes id found by
+the lookup, with this you can also verify node are near queried coordinates).
+For example, to obtain information about the nodes 1068822559 and 561188093, use
+the following command:
+
+{{{
+    echo 1068822559 561188093 \
+      | ./src/route_query_nodes_database SOMEWHERE/route_db
+}}}
+

src/Makefile

@@ -4,7 +4,6 @@ CXXFLAGS=-O2 -std=c++11
 #LDFLAGS=-g -Wall
 LDFLAGS=-O2 -fwhole-program -flto
 LDLIBS_LMDB=-llmdb -lpthread
-LDLIBS_PQXX=-lpqxx
 LDLIBS_OSMPBF=-L../../osmpbf/osmpbf -losmpbf -lprotobuf -lz
 INCLUDE_OSMPBF=-I../../osmpbf -I../../osmpbf/osmpbf/include
 
@@ -13,7 +12,7 @@ INCLUDE_OSMPBF=-I../../osmpbf -I../../osmpbf/osmpbf/include
 all: elevation_create_database elevation_query_database route_query_nodes_database route_query_lookup_database route route_create_databases_from_pbf lmdb_rewriter analyze_pbf
 
 lmdb_rewriter: lmdb_rewriter.o
-	$(CC) -static $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB)
+	$(CC) $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB)
 
 elevation_create_database: elevation.o elevation_create_database.o
 	$(CC) $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB)
@@ -21,9 +20,6 @@ elevation_create_database: elevation.o elevation_create_database.o
 elevation_query_database: elevation.o elevation_query_database.o
 	$(CC) $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB)
 
-route_create_databases: elevation.o conv_functions.o route_create_databases.o nodes_db.o lookup_db.o
-	$(CC) $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB) $(LDLIBS_PQXX)
-
 route_query_nodes_database: route_query_nodes_database.o nodes_db.o conv_functions.o
 	$(CC) $(LDFLAGS) -o $@ $+ $(LDLIBS_LMDB)
 
@@ -54,9 +50,6 @@ elevation_create_database.o: elevation_create_database.cc conf.h elevation.h
 conv_functions.o: conv_functions.cc conf.h elevation.h types.h conv_functions.h
 	$(CC) $(CXXFLAGS) -c $< -o $@
 
-route_create_databases.o: route_create_databases.cc lookup_db.h nodes_db.h elevation.h types.h conf.h
-	$(CC) $(CXXFLAGS) -c $< -o $@
-
 nodes_db.o: nodes_db.cc elevation.h types.h conv_functions.h conf.h nodes_db.h
 	$(CC) $(CXXFLAGS) -c $< -o $@
 

src/conf.h

 #ifndef H_CONF
 #define H_CONF 1
 
-#define RV_MAXIMUM_LMDB_SIZE 21474836480L // 20 GiB
+#define RV_MAXIMUM_LMDB_SIZE 1099511627776L // 1 TiB
 #define RV_ZCHAR_LENGTH 2048 // I love big numbers
 #define RV_TEMP_TABLES_PREFIX "rv_temp_"
 #define RV_NODES_DB_RELATIVE_PATH "nodes"

src/elevation_create_database.cc

@@ -3,6 +3,7 @@
 #include <string>
 #include <stdexcept>
 #include <fstream>
+#include <sys/stat.h>
 
 #include "elevation.h"
 
@@ -14,7 +15,14 @@ int main(int argc, char** argv)
         fprintf(stderr,"Usage: %s db_path srtm_files...\n",argv[0]);
         abort();
     }
-    
+
+    char zfilename[RV_ZCHAR_LENGTH];
+    snprintf(zfilename,RV_ZCHAR_LENGTH,"%s/data.mdb",argv[1]);
+    remove(zfilename);
+    snprintf(zfilename,RV_ZCHAR_LENGTH,"%s/lock.mdb",argv[1]);
+    remove(zfilename);
+    mkdir(argv[1],0755);
+
     elevation db(argv[1],true);
     
     std::list< std::pair<int32_t,int32_t> > missing_data;