Sqlitemm: C++ wrapper for sqlite3
Posted by cheshirekow in Projects on August 21, 2011
As part of writing inkbook, I decided to use sqlite3 for data storage. The C/C++ API is actually a C API, and while it is object oriented and rather intuitive, it’s just not C++. Considering that the API is very simple and, in particular, the subset of the API I wanted to use was very simple, I went ahead and wrote a quick wrapper.
Sqlitemm provides a C++ style interface to creation of database connections and statements (as in Connection and Statement are classes). Objects are reference counted using Glib::RefPtr so memory management is a bit easier.
You can find the project in my tracker , but heres an example of it’s usage:
#include <sqlitemm.h> int main(int argc, char** argv) { // open a connection to a sqlite database Glib::RefPtr<sqlite::Connection> sqlite = sqlite::Connection::create("/path/to/db.sqlite"); // we'll reuse this variable for different statements Glib::RefPtr<sqlite::Connection> stmt; // prepare a read statement stmt = sqlite->prepare("SELECT * FROM table_a WHERE field_a=?"); // bind a value to one of the parameters stmt->bind<int>(1,10); // execute the select statement stmt->step(); // read out the result set while(stmt->hasAnotherRow()) { // retrieve the first column of the result set as an integer int field_a = stmt->get<int>(0); // retrieve the second column of the result set as a string std::string field_b = stmt->get<std::string>(1); // retrieve the third column of the result set as a double double field_c = stmt->get<double>(2); // do something } // if we want to reuse the statement we need to call this stmt->reset(); // prepare a second statement, note that all memory allocated for the // first statement is released here, because the smart pointer is // reassigned and the object it points to only has one outstanding reference stmt = m_sqlite->prepare("UPDATE table SET field_a=? WHERE field_b=?" ); stmt->bind<int>(1,20); stmt->bind<std::string>(2,"something"); // now actually execute the statement stmt->step(); // note that we do not have to explicitly close the connection, when // the sqlite variable goes out of scope, the smart pointer will drop it's // reference to the underlying Connection object, and the connection // object will be destroyed. The databse is closed during the destructor // of the Connection object. return 0; } |
Texmake: a makefile generate for latex projects
Posted by cheshirekow in Uncategorized on August 18, 2011
texmake is a makefile generator, much like cmake. In fact, it originally started with me looking for ways to make cmake work with latex documents. After a bit of work, I decided the system was too much designed around the c/c++ build process, so I started working on some tools to simplify my life.
Keeping it simple
texmake tries to eliminate as much work as possible from the document writer. It will try to figure out and resolve dependencies, including graphics files which require conversion from their source format. It also allows for out-of-source builds, so that the source directory doesn’t get all cluttered with the intermediate files that latex generates.
Example
Consider this scenario: We have a very long complicated report to write (i.e. software manual, end-of-project report, thesis, how-to-book). We want to generate both a full version, and a simplified quick-start kind of version. The quick start version will contain a subset of the chapters of the full version. The quick start will be published in PDF and HTML (i.e. web-friendly). The long version will probably be a large document and we don’t really want it to be browsed on-line, but it is likely to be printed, so we’ll put it in pdf and dvi format, as well as epub for people who have e-readers.
Example Project Structure
In the process of making this document, we’ve generated many image files. Some of them are hand drawn SVGs. Some of them are generated tikz images. Some of them are diagrams drawn in DIA or plots in GNU Plot. Some of these figures are shown in multiple different chapters (because the author does not want to just refer the user back to a previous page, which is unnecessary in an electronic format, but may be more meaningful in a print format).
Furthermore, we have some things that need to only be included in each version. For instance each version should include some kind of header which tells the reader where he can find the other versions online, or where he can order them from.
Now, we can go maintain a makefile structure that manages all of this quite easily, but we will have to build it by hand. Every time we add a new chapter or image or output format, we have to go add a line to a makefile somewhere. Wouldn’t it be nice if all of that work would just happen?
Features
here are a list of features that I have implemented or am working on
- don’t rebuild the document unless it needs it
- multiple documents in the same project, potentially sharing various pieces of content
- monitor documents dependency on package, style, and class files so that document is rebuilt if these are updated
- monitor dependencies on all included files, so that if any included file is updated, the document is rebuilt
- only run bibtex if needed
- one of the database files is updated
- there are unresolved citations in the document
- rerun latex until it stabilizes
- discover and automatically convert graphics source files (like svg) to the kind (pdf)latex(ml) understands (pdf,eps,png)
- out of source builds so that sources can be in version control without complicated ignore rules
- caches values of environment variables and binary locations so that initial environment does not have to be manually set-up each time the document is built
- colorize output to make it clear where and when things get messed up
Usage
texmake relies on the presence of a texmakefile in the directory of the document you want to build. The texmakefile simply lists source files (.tex documents) and the output files that should be built from them. Multiple output files can be built from the same input file, so that the same latex source can be used to generate pdf, dvi, and xhtml versions (so-called single source builds).
texmake init is called from the build directory, accepting a single parameter being the directory that contains the root texmakefile. texmake resolves the absolute path to all the tools it needs (latex, pdflatex, bibtex, latexml, kpsewhich, …), and caches those locations. (TODO: also cache environment variables for latex and search paths for kpsewhich). It generates a makefile in that directory which contains rules for all of the output files registered, and creates directories, if they’re missing, in which the output files will be located.
texmake relies on GNU make to determine when documents need to be rebuilt. This makefile includes several other makefiles (if they exist) containing detailed dependency lists for each of the output files. If the output files have never been built, these included files do not exist, but that’s OK because it is already clear to make that it needs to build the output files.
The output files are built with texmake build. The texmake builder runs (pdf)latex(ml) with all the necessary switches to make sure that the source directory and output directory are searched for required files. The builder also scans the output of latex to build a list of all the files that latex loads (for the dependency graph) as well as all the files that are missing. If the missing files are graphics files, then it generates make rules for how to build those graphics files from similarly graphics files in the source directory (it is assumed the source file has the same name, but a different extension). This way, an svg source file can be used to generate pdf,eps,or png images depending on if the document being built is dvi,pdf, or xhtml (respectively). If the file in the source directory is already compatible with the latex that is being used, then no conversion is necessary. If the missing file is a bibliography file, then bibtex is run to generate the bibliography file. The builder also scans the output of bibtex to add bibliography database files (.bib files) to the dependency graph. Bibtex is also run if there are missing citations reported by latex (this is because bibtex will not include bibliography entries that are not cited in the latex source, so if a new citation is added, bibtex will need to be rerun). The texmake builder also scans the output of latex for rerun suggestions. It will rerun latex if it had to run bibtex, or if latex itself suggested that it be rerun (to get cross references straight, for example).
Major TODO items
- cache environment variables in init
- use diff to determine if TOC’s get updated (they will get touched by latex so make will always think they’re new) to determine of a rerun is necessary for TOCs
- add image generation rules for tikz
- consider a more modular design for making intermediates (i.e. run a program, like matlab, to generate intermediates)
Code
The texmake code is in my project tracker.
directoryWatch
Posted by cheshirekow in Programming, Projects on August 18, 2011
As I was working on texmake I decided that I didn’t want to figure out what all the possible auxilary output files would be for a latex document. Also, I’m suspecting that it will depend on what packages are included and things. Anyway, I wanted a way to just monitor the build directory and see all the files that were created while running latex. It turns out this is very easy on linux. This is a very simple program which watches a directory, and will print out any files that are created, modified, opened, closed, moved, or deleted. It prints out the file name, followed by a comma, followed by the notification event.
You can find the code in project tracker