![]() |
LIBDAR
|
PresentationThe Libdar library provides a complete abstraction layer for handling Dar archives. The general operations provided are:
Note that Disk ARchive and
libdar
have been released under the Gnu
General Public License (GPL). All code linked to libdar
(statically or dynamically), must
also be covered by the GPL. Commercial use is prohibited unless a contract has been agreeded with libdar's author.
This tutorial will show you how to
use the libdar API. Since release 2.0.0 the dar command-line executable also relies on this API, looking at it's code may provide a good
illustration on the way to use libdar, the file src/dar_suite/dar.cpp
is the primary consumer of the libdar API.
The sample codes provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen, which is also provided online. |
Let's StartConventionsLanguageDar and libdar are written in
C++, and so is the libdar API. While
written in C++, libdar is easily usable by both C and C++ code.
Access from other languages can be provided by specific bindings, like python bindings.
Header files Only one include file is required in your program to have access to libdar: #include <dar/libdar.hpp>
Libdar namespaceAll libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files: |
using
namespace libdar; get_version(....); |
or, as shown below, you can explicitly use the namespace in front of libdar symbols: |
|
Exceptions or no ExceptionThe library can be used with or
without exceptions.
For each example we will see a sample code for both ways. On the left
you will find code relying on exceptions, on the right without
exception:
|
|
|
All exceptions used by libdar inherit from the pure virtual class Egeneric. The only method you will need to know about for any exception is the get_message() call, which returns a message string describing the message (in human language). The type of the error is defined by the class of the exception. The possible exception types follow: |
class
libdar::Egeneric |
the parent class of all
exceptions (a pure virtual class) |
class libdar::Ememory |
memory has been exhausted |
class libdar::Ebug |
signals a bug, which is
triggered when reaching some code that should never be executed |
class libdar::Einfinint |
arithmetic error detected when
operating on infinint |
class libdar::Elimitint |
a limitint overflow is detected,
indicating the maximum value of the limitint has been exceeded |
class libdar::Erange |
signals a range error |
class libdar::Edeci |
signals conversion problem
between infinint and string (decimal representation) |
class libdar::Efeature |
a requested feature is not (yet)
implemented |
class libdar::Ehardware |
hardware problem is found |
class libdar::Euser_abort |
signals that the user has
aborted the operation |
class
libdar::Ethread_cancel |
A program has requested the
termination of the current thread while libdar was running |
class libdar::Edata |
an error concerning the treated
data has been encountered |
class libdar::Escript |
the script executed between
slices returned an error code |
class libdar::Elibcall |
signals an error in the
arguments given to a libdar call of the API |
class libdar::Ecompilation |
a requested feature has not been
activated at compilation time |
1 - First we *must* initialize libdar by checking the libdar version |
catch(libdar::Egeneric & e) |
libdar::U_I maj, med, min;
med <
libdar::LIBDAR_COMPILE_TIME_MEDIUM ) "we are
linking against wrong libdar" << std::endl; |
The get_version() function must
be called for
several reasons :
ger_version(maj, med, min, false)
2 - We should prepare the end right now!
|
libdar::close_and_clean()
|
In particular, closes_and_clean()
makes the necessary for memory to be released in the proper order. Not
calling close_and_clean() at the end of your program may result in uncaught exception message from libdar at the end of
the execution. This depends on the compiler, libc and option activated in libdar at compilation time.
3 - Intercepting signalslibdar by itself does not make use of any signal (see signal(2) and kill(2)). However, gpgme library with which libdar may be linked with to support asymetrical strong encryption (i.e. encryption using public/private keys) may trigger the PIPE signal. Your application shall thus either ignore it (signal(SIGPIPE, SIG_IGN)) or provide an adhoc handle. By default the PIPE signal leads the receiving processus to terminate.4 - Let's see the available features (Optional)
once we have called one of the get_version*
function it is possible to access the list of features activated at compilation
time: |
bool ea = libdar::compile_time::ea(); ();// to get the complete list of available routines check the file // src/libdar/compile_time.hpp from source package *or* // /usr/include/dar/compile_time.hpp from system header files
} |
You can do what you want with
the
resulting values, like just to displaying the available libdar
features (this is what dar -V does) or to terminating your program if you don't find a desired feature. However,
verifying that features are available is not strictly necessary because
libdar will tell you if an operation you call requires a feature that
has not been activated at compilation time, by throwing an Ecompile
exception (or returning
the LIBDAR_ECOMPILATION error
code if you are not using exceptions).
5 - User Interactiona) The generic user_interaction classTo be able to send messages to and retreive information from the user, a special class called user_interaction is defined. Simply put, user_interaction is a virtual class from which you have to derive your own class in order to provide user interaction (a GUI's graphical interaction, for example). There is four methods among which only three have to be overridden:void pause(const std::string &message); this method
is
called by libdar when the library needs a yes/no answer to a question, which question is provided by the std::string message.
The question given to pause() must be answered by returning
normally (= "true") or throwing a Euser_abort
exception if the user refused the proposition. Don't
worry about throwing an exception in your code; it will be trapped by
libdar if you don't want to manage exceptions, and are using libdar in
the "no exception" method. But if you really don't want to throw
exception from your code ignore this method (don't overwrite it in your
class) but rather consider the following one instead:
bool pause2(const std::string
&message);This is an alternative method to pause() as seen above. In place of
defining a pause() method in
your inherited class, you can redefine the pause2() method. The only
difference with pause() is
that the user answer to the question is returned by a boolean value,
your code does no more have to throw a Euser_abort exception to say "no".
Note that you must not redefine both
pause() and pause2()
void inherited_warning(const std::string &message); libdar calls
this protected method to display an informational message to the user. It is not
always a warning as the name suggests, but sometimes just normal
information.
std::string get_string(const std::string &message, bool echo); This call is
used
to get an arbitrary answer from the user. This is mainly used to get a
password from the user (when no password has been supplied for an
encrypted archive), the echo argument
indicates whether the user response should be displayed back on the screen. If echo is set to "false" the
implementation of get_string() should
hide the characters typed by the user (a password requested from the user).
user_interaction * clone() const;
A copy
operation must be implemented here. This is because libdar does not stores a
reference to the user_interaction class it receives but rather keeps an internal copy of it, so your class should also be ready to support a copy
constructor and an assignement operator. A simple implementation of this
method should be something
like this (even if you don't want to use exceptions):
|
#include <new> user_interaction
*my_interaction::clone() const |
b) Or The callback interaction class
|
(t_win *)(context)->show(x); } |
|
// creating an archive is simple; it is just
// first we need a user_interaction object, here
U_16 exception, libdar::archive *my_arch
=
nullptr, // see the progressive_report section below
exception,
// this gives the status of the call |
When
creating an archive, the just created object once the backup has
complete can be used only as reference for an isolation or for
a differential backups. You cannot use it for restoration, listing, or
comparison, because the underlying file descriptors are opened in
write only mode. An implementation which uses file descriptors in
read-write access is not possible and is not a good idea anyway. Why?
Because, for example, if you want to test the newly created archive,
using the newly created object would make the test rely on information
stored in virtual memory (the archive contents, the data location of a
file, etc.), not on the file archive itself. If some corruption
occurred
in the file you would not notice it.
So to totally complete the cycle as we do not perform archive isolation or differential backup here, we must destroy the archive
object we have just created, which will also close any file descriptors
used by the object:
|
|
if(exception != LIBDAR_NOEXCEPT) |
Here
we have been using pointers to simplify the way to describe object
terminason. Of course, it is also possible to use plain variable in
place of pointers. The archive object is destructed at end of the block:
{ libdar::archive my_arch = libdar::archive(...);
}Optional Arguments:Back on the optional arguments.
The archive constructor used above to
create an archive uses an argument of type libdar::archive_option_create. In
the above example, we called the constructor of this class directly
withing the argument list of the class libdar::archive's constructor. This has for effect to
built
an anonymous temporary object of this class. Such a "just born" object
has
all the necessary options set to their default values. If you want to use non default
options like compression, slicing, encryption, file filtering and so
on, you must
change the options to your need thanks to the appropriate method
provided by the libdar::archive_options_create class. Assuming we want to make
a
compressed an sliced archive we would use the following code:
|
// "dialog" below is the user_interaction object
|
// "dialog" below is the user_interaction object// we created earlier
U_16 exception, libdar::archive *my_arch
=
nullptr,
exception,
// this gives the status of the call |
In
the same way, each other
operation (diff, testing, extraction, merging, ...) has a specific
class that gathers the optional parameters. These classes are defined
in the file dar/archive_options.hpp you are welcome to refer to for
a complete up to date list of available fields. The advantage of this
class is to not break ascendant compatibility of the API when new
features get added, while it also improve readability of the code.
This way, the major current number '5' of the API should stay for a
longer time than previous numbers, because each new feature
implementation broke the ascendant compatibility by adding an new
argument to API calls. 7 - Testing the archive we have createdSo, as explained previously, we must create a new archive object but this time using the "read" constructor, which difers from the "create" constructor we used earlier by the nature of its arguments: |
// "dialog" below is the user_interaction object // we created earlier
my_arch = new libdar::archive(dialog,
|
// "dialog" below is the user_interaction object // we created earlier
my_arch = libdar::open_archive_noexcept( dialog,
exception,// this gives the status of the call f(exception !=
LIBDAR_NOEXCEPT) |
Now that we have opened the archive we can perform any operation on it. Let's thus start by testing the archive coherence: |
|
dialog, options, // the non default options set above
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
|
// "dialog" below is the user_interaction object // we created earlier
my_arch->op_listing(dialog, |
// "dialog" below is the user_interaction object // we created earlier
dialog,
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
Using this method, the library will performs
the listing by calling the warning()
method of the dialog object
once for each file listed. The warning text will consist of a
string for each file with the relevant information concatenated that
would need to be parsed if individual information was desired. This may
not be appropriate for your need and as such there is another way to get
listing information:
b) The flexible way:This new method gives information per directory libdar::archive::get_children_in_table(const std::string & dir) so you have to specify the directory you want to get content from. Second this method will return a list of object each one corresponding to an inode contained in that directory. Objects are of type libdar::list_entry |
// first we define the list that will receive information about the directory
std::vector<libdar::list_entry> info; |
Now we have to exploit the table info. The class libdar::list_entry provides a set of method to retreive information about the directory entry:
Here follows an simple example of use: |
std::vector<libdar::list_entry> info; std::vector<libdar::list_entry>::iterator it = info.begin();while(it != info.end()) { std::cout << it->get_name(); if(it->is_dir()) { tmp = my_arch->get_children_in_table(it->get_name()); std::cout << "directory with " << tmp.size() << " entries" << std::endl; } else std::cout << std::endl; ++it; } |
|
|
libdar::op_diff_noexcept(dialog, archive_options_diff(), // default options
files exception, // this
gives the i f(exception != LIBDAR_NOEXCEPT)
|
Simple, no? Just a note about the set_what_to_check() method argument of the libdar::archive_options_diff class. It may take several values:
10 - restoring filesRestoration of files is done by calling the op_extract method of class libdar::archive. |
|
dialog, "/tmp",
// where to restore files to archive_options_extract(), // default options
exception,// this gives the status of the call i f(exception != LIBDAR_NOEXCEPT) |
Here as we used default options, we restore all the files stored in the
archive into the directory /tmp (we also restore there the directory
structure stored in the archive), but we could also make a flat
restoration (ignore directory structure), as well as restore only some
of the files. By default too, dar asks user confirmation before
overwriting a file. You can change these options and many others using
the methods the class libdar::archive_options_extract. We will here restore all
that is under usr/lib and only files which filename ends with ".a", we
want libdar to skip file that would lead to overwriting an existing
file and also have libdar display files that have been skipped from the
restoration. Note that a specific paragraph is designated to masks further in this tutorial, so don't panic :-) |
archive_options_extract options; |
archive_options_extract options;
ret = libdar::op_extract_noexcept( dialog, "/tmp",
// where to restore files to options, // non default options set just above // no progressive report used
exception,// this gives the status of the call |
Last point about optional parameter concerns the set_what_to_check method. It serves two roles here:
|
archive_options_isolate options; |
archive_options_isolate options;
"/tmp", // where is saved the
exception, i f(exception != LIBDAR_NOEXCEPT) |
Now we have two archive
objects. my_arch is a
read-only object
created by the "read" constructor in a previous paragraph of this tutorial. You can do any operations with it,
like file restoration, file comparison, archive testing, as we have
done in the previous sections. The second archive object is my_cat which is a write only
object. It can only be used as a reference for another backup (a
differential backup) or as a reference for a subsequent catalogue
isolation (which would just clone the already isolated catalogue object
here).
Note that once closed (i.e.: object has been destructed) you can re-open the isolated catalogue using the "read" constructor and use it as a read-only object (you can then test its integrity as seen previously). So for now we will just destroy the extracted catalogue object, so that all its file descriptors get closed: |
|
if(exception != LIBDAR_NOEXCEPT) |
and we keep the my_arch
object for our last operation:12 - creating a differential backupThis operation is the same as
the
first one we did (archive creation). We will just provide the archive of reference as an optional parameter. If we had
not destroyed my_cat above,
we could have also used it in place of my_arch
for exactly the same result:
|
archive_options_create options;
nullptr); // no progressive report |
archive_options_create options;
libdar::archive *my_other_arch
=
"my_archive", //
the basename of the slices options, // the optional parameter as defined above
exception,
// thisgives the status of the call |
As previously, my_other_arch is
a write only object that we won't need
anymore. So we destroy it:
|
|
if(exception != LIBDAR_NOEXCEPT) |
We are
at the end of this first part of the tutorial, where we have
seen the general way to manipulate dar archives like dar command-line
does. But we still have an object we need
to destroy to cleanly release the memory used: |
|
if(exception != LIBDAR_NOEXCEPT) |
13 - Masks (Optional)
|
class libdar::bool_mask |
boolean mask, either always true
or false, it matches either all files or no files at all |
class libdar::simple_mask |
matches as done by the shell on
the command
lines (see "man 7 glob") |
class libdar::regular_mask |
matches regular expressions (see
"man 7 regex") |
class libdar::not_mask |
negation of another mask |
class libdar::et_mask |
makes an *AND* operator between
two or more masks |
class libdar::ou_mask |
makes the *OR* operator
between two or more masks |
class lbdar::simple_path_mask |
matches if it is subdirectory of mask or is a directory
that contains the specified path itself |
class libdar::same_path_mask |
matches if the string is exactly
the
given mask (no wild card expression) |
class
libdar::exclude_dir_mask |
matches if string is the given
string or a sub directory of it |
class libdar::mask_list |
matches a list of files defined
in a given file |
Let's play with some masks : |
// all files will be
elected by this mask libdar::bool_mask m1 = true; // m5 will now match
only files that are selected by both m2 AND m4 libdar::et_mask m5; // Frankly, it's
possible to have masks reference each other! libdar::not_mask m7 = m6; |
The idea here is not to create object manually, but to link their
creation to the action and choices the user makes from the user
interface (Graphical User Interface of your aplication, for example) Now that you've seen the power of these masks, you should know that in libdar masks are used at several places:
An exception is the archive testing operation, which has no fs_root argument (because the
operation is not relative to an existing filesystem), however the subtree argument exist to receive a
mask for comparing the path of file to include or exclude from the testing
operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks
will be compared to "<ROOT>/some/file"
when performing an archive test operation.
|
|
libdar::statistics report; libdar::deci(report.get_treated()).human();
|
Some libdar::archive methods (archive restoration, comparison, testing) also return a value of type libdar::statistics which can be used the same way to have a global summary of the operation but only once the operation has completed.
Last point all fields of libdar::statistics are not used for all operations, because they are not always meaningful (like filesystem error while doing a merging operation). Please refer to the libdar::archive class documentation (Doxygen) or to the <dar/archive.hpp> header file to have the details of class libdar::statistics used fields and their meaning in relation to each libdar::archive operation. |
For more detailed information
about the API you can build the API
documentation from the source code using Doxygen or get it online from dar home page or its mirror site.15 - Compilation & Linking
|
> cat my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc `pkg-config --cflags libdar` -c my_prog.cpp |
|
> gcc pkg-confg --libs libdar` my_prog.o -o my_prog |
Libdar's different flavorsWell, all the compilation and
linking
steps described above assume you
have a "full" libdar library.
Beside the full (alias infinint) libdar flavor, libdar
also comes in 32 and 64 bits
versions. In these last ones, in place of internally relying on a
special type (which
is a C++ class called infinint)
to
handle arbitrary large integers, libdar32 relies on 32 bits integers
and
libdar64 relies on 64 bits integers (there are limitations which are
described in doc/LIMITATIONS). But all these libdar version (infinint,
32bits, 64bits) have the same
interface and must be used the same way, except for compilation and
linking.
These different libdar
versions can
coexist on the same system, they
share the same include files. But the LIBDAR_MODE macro must be set to 32
or 64 when compiling or linking with libdar32 or libdar64
respectively. The LIBDAR_MODE macro defines the way the "class infinint" type
is
implemented in libdar, and thus changes the way the libdar headers
files are interpreted by the compiler. pkg-config --cflags will set the
correct LIBDAR_MODE, so you should only bother calling it with either
libdar, libdar32 or libdar64 depending on your need : "pkg-confg
--cflags libdar64" for example.
|
>
cat my_prog.cpp#include <dar/libdar.h> main() { libdar::get_version(...); ... } > gcc -c `pkg-config --cflags libdar32` my_prog.cpp
> gcc `pkg-config --libs libdar32` my_prog.o -o my_prog |
and replace 32 by 64 to link
with libdar64. |
16 - Aborting an OperationIf the POSIX thread support is
available, libdar will be
built in a
thread-safe manner, thus you may have several thread using libdar calls
at the same time. You may then wish to interrupt a given thread. But
aborting a thread form the outside (like sending it a KILL signal) will
most of the time let some memory allocated or even worse can lead to
dead-lock
situation, when the killed thread was inside a critical section and had not
got the opportunity to release a mutex.
For that reason, libdar proposes a set
of calls to abort any processing libdar call which is ran by a given
thread.
|
// next is the
thread ID in which we want to have
lidbar call canceled// here for simplicity we don't describe the way the ID has been obtained pthread_t thread_id = 161720; // the most simple call is : libdar::cancel_thread(thread_id); // this will make any libdar call in this thread be canceled immediately // but you can use something a bit more interesting: libdar::cancel_thread(thread_id, false); // this second argument is true for immediate cancellation, // of false for a delayed cancellation, in which case libdar aborts the operation // but produces something usable, for example, if you were backing up something // you get a real usable archive which only contains files saved so far, in place // of having a broken archive which miss a catalogue at the end. Note that this // delayed cancellation needs a bit more time to complete, depending on the // size of the archive under process. |
As seen above, cancellation can
be
very simple. What now succeeds when
you ask for a cancellation this way? Well, an exception of type Ethread_cancel
is thrown. All along his path, memory is released and mutex are freed.
Last, the exception appears to the libdar caller. So, you can catch it
to define a specific comportment. And if you don't want to use exceptions a
special returned code is used.
|
try libdar::archive *my_arch = |
U_16 ex; archive *my_arch = |
Some helper routines are
available to
know the cancellation status for a particular thread or to abort a
cancellation process if it has not yet been engaged.
|
pthread_t
tid; // how to know if the thread tid is under cancellation process ? if(libdar::cancel_status(tid)) cout << "thread cancellation is under progress for thread : " << tid << endl; else cout << "no thread cancellation is under progress for thread : " << endl; // how to cancel a pending thread cancellation ? if(libdar::cancel_clear(tid)) cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl; else cout << "too late, could not avoid thread cancellation for thread "<< tid << endl; |
Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it: |
try{ ... some libdar calls } catch(libdar::Ethread_cancel & e) { if(e.immediate_cancel()) cout << "cancel_thread() has been called with "true" as second argument" << endl; else cout << "cancel_thread() has been called with "false" as second argument" << endl; U64 flag = e.get_flag(); ... do something with the flag variable... } // what is this flag stored in this exception ? // You must consider that the complete definition of cancel_thread() is the following: // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0); // thus, any argument given in third is passed to the thrown Ethread_cancel exception, // value which can be retrieved thanks to its get_flag() method. The value given to this // flag is not used by libdar itself, it is a facility for user program to have the possibility // to include additional information about the thread cancellation. // supposing the thread cancellation has been invoked by : libdar::cancel_thread(thread_id, true, 19); // then the flag variable in the catch() statement above would have received // the value 19.
|
A last and important point about multi-threaded environment: An
object like any other variable cannot be modified or read (with the use
of its methods) without precaution from several threads at the same
time. Care must be taken to avoid this situation, and the use of Posix
mutex is recommanded in your program if you plan to let an archive
object be accessed by more than one thread. See the FAQ for more about this point. 17 - Dar_manager APIFor more about dar_manager, please read the man page where are described in
detail the available features. Note that for dar_manager there is not
a "without exception" flavor,
your
program
must be able to handle exceptions, which by the way are the same as the
ones described above.
To get dar_manager features you
need to use the class
database
which is defined in the libdar/database.hpp
header file so you first need to include that file. Most of the methods of the database class do use options. For the same reason as previously seen for archive manipulation, these options are passed thanks to a container class. These container classes for options used by the database class are defined in the libdar/database_options.hpp file. Let's see the
different method of the class database
:
Database object constructionTwo constructor are available: |
#include
<dar/database.hpp> |
So far, this is not much
complicated.
You can build an empty database
from nothing, or load a database to memory from a file using the second
constructor. As you can see over the filename to give in this later
constructor, we need a user_interaction
object to be able to inform the
user of any problem that could be met, and an object of class
database_open_options. This last object contains options to use for
this call (options are set to their default unless modified
explicitely). Currently, the only available option is the "partial"
option which is a boolean argument:
In all the available methods for class database, some require to load the whole database in the memory while some other only require the database header. Loading just the database header is much faster than loading the whole database, of course, and as you guess it requires much less memory. While you can perform any operation with a full loaded database, only a subset of available method will be available with a partially loaded database. If you try a method that requires a completely loaded database, you will get an exception if the object you use has been loaded with "true" as last argument (called "partial") of the constructor, and of course an empty database (built with the first constructor) is a completely loaded database, so you don't have restriction in using a new database object. But now let's see the available method for that class: Database's methods
First we will see methods that work with both partially and completely
loaded databases: |
ThanksI would like to thank Wesley
Leggette and Johnathan Burchill for having
given their feedback and
having done grammar corrections to this document. Out of this document,
I would also like to thanks them a second time for their work around
dar and libdar (Johnathan is the author of kdar, the KDE front-end for
dar).
Regards, Denis Corbin. |