Apache Thrift 0.3.0 released

by fkollmann 8/10/2010 1:10:56 PM

The Apache Thrift v0.3.0 RTM has been released after RC6 has been approved.

Thrift is a multi-platform communication framework: http://incubator.apache.org/thrift/ . The framework is also used by Apache Cassandra.

The binaries provided here include:

  • Thrift compiler (cygwin)
  • Thrift .NET 2.0 library
  • Thrift .NET 3.5 library (using HashSet)

DOWNLOAD

Apache Thrift 0.3.0 RC4 released

by fkollmann 6/16/2010 4:37:04 PM

The new Thrift v0.3.0 RC4 is available (for a week now). Since the team seems to take some more time to get the version ready which will most likely by the RTM, I would like to provide the Windows binaries for that build:

Thrift is a multi-platform communication framework: http://incubator.apache.org/thrift/ . The framework is also used by Apache Cassandra.

DOWNLOAD (Win32 compiler, .NET 3.5 library, .NET 2.0 library)

OUTDATED: see http://www.fkollmann.de/v2/post/Apache-Thrift-030-released.aspx

Thrift Windows Binaries for Cassandra

by fkollmann 4/18/2010 5:56:44 PM

UPDATE: Binaries for newer versions can be found here: http://www.fkollmann.de/v2/post/Apache-Thrift-030-RC4-released.aspx .

I have been playing around with Apache Cassandra lately which is now available in v0.6. Unfortunately it’s hard to find Windows binaries for the required Apache Thrift compiler v0.2. Since I had to spend several hours to compile the compiler using cygwin, I would like to share the binaries:

Download: Thrift Compiler v0.2 for Windows

 

UPDATE: Bernhard Glück also provided binaries in the mean time.

Generating time-based/type 1 GUIDs

by fkollmann 3/31/2010 5:25:10 PM

Unfortunately I have been unable to find any implementation for time-based GUIDs (or “type 1” to call them correctly). Usually they are not used that often today since the random generated once are quite as good (see .NET System.Guid implementation).

However since I have been doing some testing with the Apache Cassandra “NoSQL” database I needed an implementation for .NET. This is mainly because data is sorted on write (one of the big differences to RDBS) by it’s key name AND only by it’s key name. This means if you have a distributed system writing in a database and you need a time based sorting which is globally unique, then time-based GUIDs are your choice.

I did the following implementation and ran some test cases on it. It should be fine (esp. since this is not the first time I implement them):

namespace System
{
    /// <summary>
    /// helper class to create "type 1"/time based GUIDs.
    ///
    /// see
http://de.wikipedia.org/wiki/Universally_Unique_Identifier
    /// see http://tools.ietf.org/html/rfc4122
    /// </summary>
    public static class TimeGuid
    {
        public static Guid New()
        {
            var id = new byte[16];

            // set timestamp and version
            var ts = GetNextCurrentTimestamp();

            id[0] = (byte)(ts & 0xff);
            id[1] = (byte)((ts >> 8) & 0xff);
            id[2] = (byte)((ts >> 16) & 0xff);
            id[3] = (byte)((ts >> 24) & 0xff);

            id[4] = (byte)((ts >> 32) & 0xff);
            id[5] = (byte)((ts >> 40) & 0xff);

            id[6] = (byte)((ts >> 48) & 0xff);
            id[7] = (byte)(((ts >> 56) & 0x0f) | 0x10 /* version number: 1 */);

            // set clock sequence
            var clock = _clockSequence;

            id[8] = (byte)(((clock >> 8) & 0x03) | 0x08);   // force two highest bits to be "10"
            id[9] = (byte)(clock & 0xff);

            // set mac address/node id
            var nodeId = _nodeId;

            if (nodeId.Length < 6)
                throw new NotSupportedException("node id is too short; no network card found?");

            id[10] = nodeId[0];
            id[11] = nodeId[1];
            id[12] = nodeId[2];
            id[13] = nodeId[3];
            id[14] = nodeId[4];
            id[15] = nodeId[5];

            return new Guid(id);
        }

        private static readonly short _clockSequence = 0;
        private static readonly long _timeStampBase;
        private static readonly byte[] _nodeId;
        private static long _lastTimeStamp;

        /// <summary>
        /// contains the error occurred when trying to
        /// determine the node id.
        ///
        /// consider this is a warning, there is a fallback.
        /// this field containing an error does not lead
        /// to invalid GUIDs
        /// </summary>
        public static Exception NodeIdError
        {
            get; private set;
        }

        static TimeGuid()
        {
            // create base timestamp; in .NET ticks are 100ns
            // which conforms the GUID standard
            _timeStampBase = new DateTime(1582, 10, 15, 0, 0, 0, 0, DateTimeKind.Utc).Ticks;

            // initialize current timestamp
            _lastTimeStamp = (DateTime.UtcNow.Ticks - _timeStampBase);

            // initialize clock sequence with a random value
            var rg = new Random(DateTime.Now.Millisecond);
            var rnd = Guid.NewGuid().ToByteArray();

            _clockSequence = (short)(
                (ushort)rnd[rg.Next(0, 15)] |
                ((ushort)rnd[rg.Next(0, 15)] << 0xff)
                );

            // gather node id
            _nodeId = GetNodeId();
        }

        private static long GetNextCurrentTimestamp()
        {
            var cts = (DateTime.UtcNow.Ticks - _timeStampBase);
            var lts = Interlocked.Read(ref _lastTimeStamp);

            // update last timestamp because it's outdated
            if (cts > lts)
            {
                if (Interlocked.CompareExchange(ref _lastTimeStamp, cts, lts) == lts)
                    return cts;

                // the last timestamp was already updated
                // so this means it's now up to date...
                // simply rely on its safe value!
                return Interlocked.Increment(ref _lastTimeStamp);
            }

            // the last timestamp is newer or
            // yet unchanged; rely on its safe value!
            return Interlocked.Increment(ref _lastTimeStamp);
        }

        private static byte[] GetNodeId()
        {
            try
            {
                // gather all network interfaces
                var nics = NetworkInterface.GetAllNetworkInterfaces();

                foreach (var n in nics)
                {
                    // check if a network interface has
                    // a mac address which is long enough
                    // (older devices have 6 bytes which is
                    // sufficient for GUIDs)
                    var mac = n.GetPhysicalAddress();

                    var macAddr = mac.GetAddressBytes();

                    if (macAddr.Length >= 6)
                        return macAddr;
                }
            }
            catch (Exception x)
            {
                NodeIdError = x;

                return Guid.NewGuid().ToByteArray();
            }

            // not supported or no network card found?
            NodeIdError = new Exception("no network card found");

            return Guid.NewGuid().ToByteArray();
        }
    }
}

Feel free to use the code like you want. If you find errors or good points for improvement, please let me know.