language agnostic - Does there exist an open-source distributed logging library? -


i'm talking library allow me log events different machines , align these events on "global" time axis sufficiently high precision.

actually, i'm asking because i've written such thing myself in course of cluster computing project, found terrifically useful, , surprised couldn't find analogues.

therefore, point whether exists (and better contribute it) or nothing exists (and better write open-source analogue of solution).

here features i'd expect such library:

  • independence on clock offset between different machines
  • timing precision on order of @ least milliseconds, preferably microseconds
  • scalability thousands of concurrent logging processes, @ least several megabytes of aggregated logs per second
  • soft real-time operation (t.i. don't want collect 200 big logs 200 machines , compute clock offsets , merge them - want see happens "live", perhaps small lag 10s)

facebook's contribution in matter called 'scribe'.

excerpt:

scribe server aggregating streaming log data. designed scale large number of nodes , robust network , node failures. there scribe server running on every node in system, configured aggregate messages , send them central scribe server (or servers) in larger groups.

...

scribe implemented thrift service using non-blocking c++ server. installation @ facebook runs on thousands of machines , reliably delivers tens of billions of messages day.

the api thrift-based, have platform coverage, in case you're looking simple integration java may want have @ digg's log4j appender scribe.


Comments

Popular posts from this blog

android - Spacing between the stars of a rating bar? -

aspxgridview - Devexpress grid - header filter does not work if column is initially hidden -

c# - How to execute a particular part of code asynchronously in a class -