I’d like to see how deep the rabbit’s hole is, starting it with a quick round of Java queues. The most important one in low latency application context is the circular queues or ring buffers. These are the same. It’s a fixed size data buffer, connecting the end to the beginning creating a nice infinitive data ring, so your hamster could easily run around and around and around…
Another important architectural point of circular buffer is that it’s a FIFO (First In First Out) construction, easy data piping.
Major points of a low latency application:
- Lock Free
- Wait Free
Some major implementations
- Fast Flow: FastFlow is a parallel programming framework for multicore platforms, which uses hybrid single-producer/single-consumer queues as a base underlying component for inter-thread communication
- Thompson queue
- Lamport queue: http://en.wikipedia.org/wiki/Lamport’s_bakery_algorithm
A possible implementation of this queue in Java is using an array based solution. It needs to enqueue the new items into the structure and to dequeue from there. It also needs to shift the data to the dequeued space in the array. Because it has always a fixed array size, it needs a starting array size and the implementation must be able to expand the size if it’s necessary.
I’m thinking about building a demo TX site, playing around low-latency problems with Java. I need to make some researches for this journey so I decided to create a development check-list what are the main points of a low-latency web application. I won’t really deal with front-end side, I’ll create a RESTful API, that will be enough. It should be a websocket driven communication form.
- using non blocking multi-threaded technologies
- avoid the complex and unnecessary frameworks (plain jdbc vs JPA?)
- reducing memory usage
- check java 8 related parts
- using a highly responsive server (Netty)
I’m not sure which one was the first when I met this reactive idea. Maybe Odersky’s Principles of Reactive Programming was that. It’s a really great but not too easy Coursera session with Scala, of course. And there is the core, The reactive manifesto. You can signed it if agree with this. Yes, I signed it.
It’s all about the standard vertical and horizontal scaling stuff. The main idea contains 4 points:
- react to events (Event driven programming)
- react to load (scalability)
- react to failure (resilient systems)
- react to users (responsive)
Best implementation of Reactive manifesto is Akka toolkit.
It was a crackin’ meetup. Alex Dean talked about an open source event analytics platform. It’s built on Scalding as a MapReducer with Scalaz, Spray and ScalaCheck. These are the main scala technologies in the project. Alex talked about the business model. So it’s an open source project and as usual there is a business support on the Snosplow’s side.
There is the GitHub page: https://github.com/snowplow/snowplow
I have to paste the scalding’s logo, because that’s really cool:
As a well mannered computer person I enjoy every documentary movies about computer science. I’ve seen this BBC mini series called All Watched Over by Machines of Loving Grace this weekend. Huge amount of economics and philosophy mixing with dusty computers. We are just part of the whole system, we know that since Matrix.
Just a quick blog about Yahoo’s Hadoop cluster. Here it is, pretty awesome:
I’m going to be a Meetup junkie. London is brilliant and full of tech meetups. I’ve missed some London Java Community events in the last few weeks. It was a huge mistake. Luckily I’ve cached the last one. Meet a Project was the event’s name. There we had Mani who talked about OpenJDK related community projects like Adopt OpenJDK and Adopt a JSR. I don’t know which one is the more important. I’m trying to figure out. Dealing JSRs looks more exciting for me.
The London Java Community has an important part of the Adopt a JSR project. Here is a list of them: https://java.net/projects/ljc-london-jug/pages/AdoptAJSRProgram
I’d like to choose some of them, driving inside these JSRs, and helping the community. As Mani’s mentioned it: This programme impacts ~10 million developers & over 1 Billion end users!