blip.kummerlaender.eu

» Interpose open library function

open is not as side effect free as I had imagined - i.e. if the flag O_TRUNC is passed it truncates the file contents alongside opening the file descriptor. In practice this is done by emacs prior to writing the new file content and as such needs to be intercepted so we can start tracking the file before it is changed.

Interposing open required some changes to make the library work without including fcntl.h. This header not only defines some of the flags we require to check if a library call actually is able to change files but also defines the open library function.

While implementing this change I noticed that the function interpositions implemented in C++ actually need to be declared as external "C" so their names do not get wrangled during compilation. I suspect that this was previously implicitly done for e.g. mmap and write by the included C standard library headers. However this did not work for open which is why all function interpositions are now explicitly declared external.

End result: emacs file changes are now tracked correctly.

» Implement static allocator for initialization

The previous interposition logic based on plain usage of dlsym analogously to various online examples led to a deadlock during neovim startup. This deadlock was caused by neovim’s custom memory allocation library jemalloc because it calls mmap during its initialization phase. The problem with calling mmap during initialization is that this already leads to executing libChangeLog’s mmap version whoes static actual_mmap function pointer is not initialized at this point in time. This is detected and leads to a call to dlsym to remedy this situation. Sadly dlsym in turn requires memory allocation using calloc which leads us back to initializing jemalloc and as such to a deadlock.

I first saw this as a bug in jemalloc which seemed to be confirmed by a short search in my search engine of choice. This prompted me to create an appropriate bug report which was dismissed as a problem in the way mmap was interposed and not as a bug in the library. Thus it seems to be accepted practice that it is not the responsibility of a custom memory allocator to cater to the initialization needs of other libraries relying on function interposition. This is of course a valid position as the whole issue is a kind of chicken and egg problem where both sides can be argued.

To cut to the chase I was left with the only option of working around this deadlock by adapting libChangeLog to call dlsym without relying on the wrapped application’s memory allocator of choice. The most straight forward way to do this is to provide another custom memory allocator alongside the payload function interpositions of mmap and friends.

init/alloc.cc implements such a selectively transparent memory allocator that offers a small static buffer for usage in the context of executing dlsym.The choice between forwarding memory allocation requests to the wrapped application’s allocator and using the static buffer is governed by init::dlsymContext. This tiny helper class maintains an dlsym_level counter by posing as a scope guard.

The end result of this extension to libChangeLog is that it now also works with applications using jemalloc such as neovim and should overall be much more robust during its initialization phase.

» Implement support for excluding arbitrary paths from tracking

The library may be provided with a new-line separated list of regular expressions via the newly introduced CHANGE_LOG_IGNORE_PATTERN_PATH.

Any proposed tracking path that is matched by any of the provided patterns is excluded from change reporting. This functionality uses the Standard’s regular expression parsing functionality and as such doesn’t introduce any new dependencies. If no file path is provided or the provided file path is unreadable all paths will be tracked.

change was adapted to set CHANGE_LOG_IGNORE_PATTERN_PATH to .change_log_ignore which means that it will by default exclude any patterns provided via this file in the current working directory.

An example for such a file customized for hiding vim’s internal write logic may look as follows:

[0-9]+
[^~]*~
[.*\.viminfo
.*\.swp

Note that this is implemented in a fashion where it is not guaranteed that the full canonical path is checked against the patterns. It remains to be decided if this is enough for all common use cases of this new functionality.

tracking::PathMatcher lacks any explicit thread synchronization - according to my current knowledge this should not be necessary as we are only ever reading the private std::vector<std::regex> instance. If invalid regular expressions are provided they are silently ignored.

» Reimplemented TakeWhile and DropWhile in terms of ListIndex

• new higher order list operation ListIndex returns the index of the first list element satisfying a given predicate function
  • this enables finding the split index required for both TakeWhile and DropWhile
  • actual list manipulation is then performed using lower order Take or Drop
  • analogously to list-index in SRFI-1
  • added appropriate test case
• added higher order predicate utility function utility::predicate_assurance
  • as its name implies this function enables assuring that a value satisfies a given predicate
    • if this is not the case a surrogate value is returned instead
  • this is used to return a appropriate size value if the ListIndex call fails because no element satisfies its predicate
  • detail::predicate_negator was moved from Remove’s implementation details into the newly introduced utility namespace
    • as it is required by both TakeWhile and DropWhile in addition to Remove
• continuation of 8e49cc6

» Revamped partial function application

• moved internals into separate header i.e. the detail namespace relating to Apply
• implemented automatic alias selection by implementing aliae of the basic variadic type template alias in different base classes
  • variadic partial application is implemented in detail::apply_variadic
    • detail::apply_single and detail::apply_pair define aliae to detail::apply_variadic’s type template alias
    • both restricted aliae derive from detail::apply_variadic
  • Apply derives from any of the aliae defining base classes depending on the count of placeholders as determined by detail::coun t_placeholders
    • Apply is guaranteed to always be derived from detail::apply_variadic one way or the other
• changed functions, test cases and examples depending on Apply accordingly
  • Length had to be reimplemented without Apply as it doesn’t allow usage of aliae expecting a different count of arguments anym ore
    • this is a advantage in the sense that core functionality of TypeAsValue now doesn’t depend on this complex partial application implementation anymore
    • such functionality may be reimplemented separately from Apply
• removed unnecessary tav namespace prefixes