A basic atomic number implementation

A basic atomic number implementation in python.

from datetime import datetime, date
from functools import wraps
import threading



def synchronized(function):

    def synched_function(self, *args, **kwargs):
        function._lock__ = threading.Lock()
        with function._lock__:
            return function(self, *args, **kwargs)
    return synched_function


class AtomicLong:
    def __init__(self, num):
        self._num = num

    @synchronized
    def increment_and_get(self):
        self._num += 1
        return self._num

    @synchronized
    def add_and_get(self, val):
        self._num += val
        return self._num

    @synchronized
    def set_value(self, val):
        self._num = val


class IdGenerator:
    RESET_MARKER = 101
    MAX_SEQUENCE_VALUE = 9999
    ATOMIC_LONG = AtomicLong(RESET_MARKER)

    @staticmethod
    def generate_id():
        # some business logic here
        sequence_number = IdGenerator.ATOMIC_LONG.increment_and_get()
        first_day = date(date.today().year, 1, 1)
        today = date.today()
        diff_days = (today - first_day).days
        year = date.today().year % 100
        seconds_passed_since_midnight = int(
            (datetime.now() - datetime.now().replace(hour=0, minute=0, second=0, microsecond=0)).total_seconds())

        id = "{0}{1}{2}{3}{4}-{5}".format("A", "O", diff_days, year,
                                                   seconds_passed_since_midnight,
                                                   sequence_number)
        print(id)
        if sequence_number > IdGenerator.MAX_SEQUENCE_VALUE:
            IdGenerator.ATOMIC_LONG.set_value(IdGenerator.RESET_MARKER)
        return id



if __name__ == '__main__':
    id_gen = IdGenerator()
    thread1 = threading.Thread(target=id_gen.generate_id)
    thread2 = threading.Thread(target=id_gen.generate_id)
    thread3 = threading.Thread(target=id_gen.generate_id)
    thread1.start()
    thread2.start()
    thread3.start()

Pagination Using Redis

Continuing my last post on Redis, I was trying to achieve simple pagination using Redis. A use case can be showing reviews/comments on a web page.

When using a sorted set –

public static  void testRedisPagination(){
    ShardedJedis redis = JEDIS_POOL.getResource();
    redis.del("temp");
    String redisKey = "temp";
    for (int i = 0; i < 38; i++) {
        String val = "val - " + i;
        redis.zadd(redisKey, i, val);
    }
    long total = redis.zcard(redisKey);
    System.out.println("Total value inserted - " + total);
    int perPage = 7;
    System.out.println("Total number of pages - " + total/perPage);
    for (int pageNumber = 1; pageNumber <= total / perPage + 1; pageNumber++) {
        long start  = total - ((pageNumber-1)*perPage + 1);
        long end = start - perPage;
        System.out.println(" ===== Printing page " + (pageNumber));
        Set<String> values = redis.zrevrangeByScore(redisKey, start, end + 1);
        for (String value : values) {
            System.out.println(value + "\t");   
        }
        System.out.println("\n\n");
    }

When using a list

public static void testRedisPagination(){
 ShardedJedis redis = JEDIS_POOL.getResource();
 String redisKey = "temp";
 for (int i = 0; i < 38; i++) {
    String val = "val - " + i;
    redis.lpush(redisKey, val);
 } 
 long total = redis.llen(redisKey);
 System.out.println("Total value inserted - " + total);
 int perPage = 5;
 System.out.println("Total number of pages - " + total/perPage);
 for (int pageNumber = 1; pageNumber <= total / perPage; pageNumber++)  {
   int start = (pageNumber-1)*perPage + 1;
   int end = start + perPage -1;
   System.out.println(" ===== Printing page " + (pageNumber));
   List<String> values = redis.lrange(redisKey, start-1, end-1);
   for (String value : values) {
     System.out.println(value + "\t"); 
   }
 System.out.println("\n\n");
}

Using Jedis

Jedis is one of Redis clients written in Java, though I have not tried using any other client. It is very easy to configure Jedis and use it as a client. Lets quickly jump over to the code part and understand things.
Using Maven dependency:
<dependency>
  <groupId>redis.clients</groupId>
  <artifactId>jedis</artifactId>
  <version>2.6.0</version>
</dependency>
Problem Statement: – We have to add items to a sorted set on basis of some random scores attached to the values. Also there has to be maximum of 10 items present in the sorted set. We will use the functions zadd and zremrandeByScore.
Some Java code:
 Initializing the Redis Client – Our server is up on localhost:6379 (default). For multiple servers running we can add them to pool.
public RedisCacheManager() {
    List<JedisShardInfo> shards = new ArrayList<JedisShardInfo>();
    for (String host : HOSTNAMES) {
        JedisShardInfo si = new JedisShardInfo(host, Constants.CACHE_PORT);
        shards.add(si);
        LOGGER.info("Added host to shard: " + host);
    }
    GenericObjectPoolConfig jedisConfig = new GenericObjectPoolConfig();
    JEDIS_POOL = new ShardedJedisPool(jedisConfig, shards);
}
Pushing items in a sorted set –
public boolean pushActivity(String activity, String activityId, double score, int ttl) {
    ShardedJedis redis = JEDIS_POOL.getResource();
    String redisKey = null;
    try {
        redisKey = activityId;
        redis.zadd(redisKey, score, activity);
        if (ttl > 0) {
            redis.expire(redisKey, ttl);
        }
        Long count = redis.zcount(redisKey, Double.MIN_VALUE, Double.MAX_VALUE);
        if (count > 10) {
            redis.zremrangeByScore(redisKey, Double.MIN_VALUE, count - 10);
        }
        return true;
    } catch (Exception e) {
        LOGGER.error("Unable to enter data into redis for - " + redisKey + " with activity - " + activity, e);
    } finally {
        JEDIS_POOL.returnResource(redis);
    }
    return false;
}
Retrieving item from Redis –
public Set<String> retrieveItem(int activityId, int maxIndex) {
    ShardedJedis redis = JEDIS_POOL.getResource();
    String redisKey = activityId;
    Set<String> redisValue = Collections.emptySet();
    try {
        redisValue = redis.zrange(redisKey, 0, maxIndex);
    } catch (Exception e) {
        LOGGER.error("Unable to fetch data from redis for - " + redisKey, e);
    } finally {
        JEDIS_POOL.returnResource(redis);
    }
    return redisValue;
}

Using Spring with Hibernate and c3p0 Connection Pool.

C3P0 is a very nice tool to manage database connections. I had hard time configuring Apache DBCP/2 so tried c3p0. There are many config options to set and the setting has to be done carefully so that we do not end up choking our database. Let us understand some of the config options.
  • testConnectionOnCheckin validates the connection when it is returned to the pool.
  • testConnectionOnCheckOut would ensure active connections before use, would be too expensive to do.
  • idleConnectionTestPeriod sets a limit to how long a connection will stay idle before testing it. Without preferredTestQuery, the default is DatabaseMetaData.getTables() – which is database agnostic, and although a relatively expensive call, is probably fine for a relatively small database. If you’re paranoid about performance use a query specific to your database (i.e. preferredTestQuery="SELECT 1")
  • maxIdleTimeExcessConnections will bring back the connectionCount back down to minPoolSize after a spike in activity, the connection is removed from the pool and returned back to db.
  • numHelperThreads it will help c3p0 spawns helper threads to manage the connections and returning them back

My spring configuration goes as –

<bean id="dataSource"  class="com.mchange.v2.c3p0.ComboPooledDataSource">
    <property name="driverClass">
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property name="jdbcUrl">
        <value>jdbc:mysql://localhost:3306/api</value>
    </property>
    <property name="user">
        <value>root</value>
    </property>
    <property name="password">
        <value></value>
    </property>
    <property name="idleConnectionTestPeriod">
        <value>300</value>
    </property>
    <property name="maxIdleTimeExcessConnections" value="180"/>
    <property name="maxPoolSize">
        <value>100</value>
    </property>
    <property name="acquireIncrement">
        <value>1</value>
    </property>
    <property name="maxStatements">
        <value>0</value>
    </property>
    <property name="minPoolSize">
        <value>10</value>
    </property>
    <property name="unreturnedConnectionTimeout">
        <value>3600</value>
    </property>
    <property name="preferredTestQuery">
        <value>SELECT 1</value>
    </property>
    <property name="initialPoolSize">
        <value>10</value>
    </property>
</bean>
Things to keep in mind:
  • To get an idea, please try to check the number of connection the app has hooked up with the database. For MySql try –  SHOW STATUS WHERE variable_name = ‘Threads_connected’;
  • When using Hibernate try to take care of opening and closing of sessions. If sessions are not properly closed, the connections are not freed and eventually it will choke the database.
Further Read - How to configure c3p0

Jackson JSON parser – Writing Custom Deserializer

One of major difference between Jackson and GSON is how they handle json parsing and mapping the json to a class object. While doing so Jackson uses strict checking which means the object from which the json was created and the class to which the json has to be mapped should be in strict sync.

Consider the following example –

class Foo {
private String value;
}
class Bar {
private String value;
}

and

String json = "{\"value\" : \"whatever\"}";
new Gson().fromJson(json, Foo.class);
new Gson().fromJson(json, Bar.class);

Gson is setup to perform a best effort to deserialize the given JSON into an instance of the given class. It will map as many fields as it finds. If none are found, that’s too bad, mapping will not be done.

Coming to the custom deserializer thing. We may need to write one in few cases as – changing date format of the json string to our custom one. For writing our custom deserializer we need to write a class which extends JsonDeserializer<T>. In this class we implement  the method deserialize(…) which contains our custom conversion code.

public class CustomJsonDateDeserializer extends JsonDeserializer<Date> {
@Override
public Date deserialize(JsonParser jsonParser, DeserializationContext context)
throws IOException, JsonProcessingException {
SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm a z");
String date = jsonParser.getText();
try {
return format.parse(date);
} catch (ParseException e) {
throw new RuntimeException(e);
}
}
}

Now annotate the field in the POJO which has to be deserialized using the custom deserializer.

@JsonDeserialize(using = CustomJsonDateDeserializer.class) private Date createdAt;

That’s it. Pretty simple thing.

RestEasy 3.0

We were using RESTEasy 2.0 in on of our earlier projects. RESTEasy is a JBoss project that provides various frameworks to help you build RESTful Web Services and RESTful Java applications. The 2.0 version was quite simple to use. I decided to use version 3.0 in one of recent projects and the migration was not that tough. A lot of chages have been done in the newer version. I am going to mention the ones which I found out.

RESTEasy Client 

In version 2.0

      String url = "http://resetservice.com/api/testService";
      ClientRequest request = new ClientRequest(url);
        request.body("application/json", json);
        response = request.post();
        int status = response.getStatus();
        if (status == 200) {
            String data = (String) response.getEntity(String.class);
            Boolean value = new Gson().fromJson(data, Boolean.class);
            return value;
        }
        return false;

The same thing  in 3.0 now changes to –

     ResteasyClient client = new ResteasyClientBuilder().build();
     String result = getResponseString("http://resetservice.com/api/testService");
     Boolean returnVal = clientUtil.getObjectFromJson(Boolean.class, result);
     return returnVal;
Authentication

I also had to use some in-house authentication technic.  I used it in very basic way. I checked for a particular request header (authorizationString) for authenticating the request. If that header is not present the request gets aborted. Below is the code snippet for the same.

@Provider
@ServerInterceptor
public class AuthorizationRequestFilter implements ContainerRequestFilter {
@Override
public void filter(ContainerRequestContext requestContext) throws IOException {
    final SecurityContext securityContext =requestContext.getSecurityContext();
    String header = requestContext.getHeaderString("authorizationString");
    try {
         if (securityContext == null || header == null || header == "" ||
                                (header != null && !header.equals(
                                 ConfigReader.getInstance().getAuthToken()))) {
             requestContext.abortWith(
             Response.status(Response.Status.UNAUTHORIZED).entity("User cannot access the resource.").build());
          }
        } catch (ConfigurationException e) {
          e.printStackTrace();
     }
   }
}

Amazon RDS with MySql – How to create an instance!

This post will guide through how to use Amazon RDS, the steps of creating an instance and other related things.

Before going further one can take a look on few links to get a hang of what RDS is all about and how does it perform against other available solutions.
Lets get started.
1. Launch DB instance – AWS Services are relatively cheaper in USA region, so you can consider about choosing the region. Go to Amazon AWS console and select RDS from there. You will land up on the below page. Click “launch a DB Instance”.

2. Select mysql engine.
3. Production- Choose Multi-AZ Deployment.
4. Specify DB Details – On the basis of your requirements you can choose DB Instance Class. In my case I used db.t2.medium which was of 5GB capacity. Specify user and password for the db from here.
5. Configure Advanced Settings – You can define your custom parameter group so you can simply choose default parameter group here. If you have a pre-defined security group, you should choose that one here. 
6. DB Instance Created- The database has now been created. It may take couple of minutes for the db to set up completely.
7. The newly created DB instance will start reflecting in the dashboard.

Changing the default configuration variables of MySql

Now after the db instance has been created we need to change a bit of configurations and monitor the db.

I struggled a bit when I had to change the default configuration values. To change the config values (equivalent to changing values in my.cnf file) we have these following steps.
1.  Go to RDS dashboard and select the database which you have created. Choose “Parameter Group” from the left pane.
2. Click on “Create DB Parameter Group” to create a custom Parameter Group. Choose some suitable name and description.
3. Now choose “Edit” to change the config. Some important variables as – max_connections, query_cache_size etc can be changed from here. The changes will be applied and reflected.

Monitoring

One of the coolest features of RDS is its monitoring. You can check out for n different metrics. – CPU Utilizations, Number of connections etc.
Simple, isn’t it? Feel free to comment.

A Bong Land Away From Bengal

Chittaranjan Park (Bengali: চিত্তরঞ্জন পার্ক), aka C.R. Park is a neighborhood in South Delhi, and home to a large Bengali community in Delhi. That’s a very known fact. People say, if you want to get a feeling of how Durga Puja is celebrated in West Bengal, then go to this place. I visited this place on the last day of Puja and witnessed the show and it was totally worth it. 


The Bengalis go little insane during Puja period. They have their own way of celebrating it. Seems like there is a sudden induction of too much happiness in these people. And thanks to whatever it is, the place turns into an electric one during the Puja. The streets were decorated with light bulbs – not too bright.  Too many people walking the streets. Kids shouting, girls giggling and ladies blabbering. Too much of chit-chat and I could not hear myself at times. Initially I was little disturbed by the noise, but after I walked enough along with the crowd I start enjoying it. And then there were things around which kept me engaged. The sound of drums, flock of girls in ethnic dresses dancing on a stage nearby the Pandal, a beautiful woman in red embroidered sari singing some classic Bangla song, young girls chatting shyly with boys near the food stalls, smell of fish in and around the Pandal. Everything was oozing out of itself some kind of Bengali class. 

Though I am not a great fan of Bengal/i, but one thing I must mention that I like is – they carry their culture wherever they go. You must be having some Bengali family living near by and they having your mind blown off with some great foods or at least some great variety of sweets. They make you feel the impact of rich culture own and they belong to. A single Bengali family can form community of its own and then slowly engulfing the people near by. A small place secluded from the typical charm which Delhi carries was what the place appeared. The streets reminded me of  a typical Kolkata streets minus the crowd part. It was indeed a nice experience. 

It Was Dreddfully Fantastic



Cast: Karl Urban, Olivia Thirlby, Lena Headey, Wood Harris
Director: Pete Travis

For a long time we all have been waiting for a perfect action movie where there is no Super-hero but a bad-ass toughie. Lessons, protagonism and philosophy ruin the essence of action. A perfect action flick is the one where we get some gory stuffs, gunshots, some dark humor, rushing sequences and  a music to elevate all these up to a different level. With Dredd you are guaranteed to get everything. A perfect violence crime thriller with every element presented in pristine form. Right from the start, the movie assures you that you will be fed a great dose of insanity and in the end you have with you a head swirling on the tune of it.

The movie, set in future,  revolves around busting of a ghetto gang homed in a multi-storeyed led by an unattractive blood crazed lady called Ma-Ma. This gang deals in production and selling of drugs and their main product- Slo Mo – which slows the perception of time of the addicts. The slow motioned effects of this drug has been so wonderfully portrayed. Everything stops and you feel yourself to be the subject. Judge Dredd, character led by Karl Urban, is a ruthless executioner who believes in law and exterminates every one who goes against it. He teams up with a rookie who is a mind-reader too and very strategically uproots the gang.

The plot may not be the greatest, but it does not have to be. What matters the most is the presentation. And for that the movie wins a standing ovation. Cinematography, I must mention, is breathtaking – as good as it should be. The futuristic city – chaotic, gloomy and plagued by social malaise – speaks off its story. The slow motion acts are one of many great things about the movie. Who needs a description on acting now? Karl Urban is such a perfect fit for the lead role. With laconic dialogues in husky voice, the helmeted actor will surely win your heart. The character(of a rookie) played by Olivia Thirlby is very powerful. She is on a par with Dredd – witty and equally intelligent. The music is just divine. It keeps you on your toes every time. The thrilling sequences coupled with great background music will make your heart slip at times.

Dredd is what we have been looking for so long. It has been crafted very nicely. A perfect juxtaposition of brutality and violence! Leave everything behind and behold to those bloody sequences. It would very foolish mentioning what 3D effects have done to the movie. For those who enjoyed Kill Bill and District 13, here is Dredd satisfying the hunger of both.

The Greatest Fear Of All – Fear Of Death


No one knows whether death may not be the greatest of all blessings for a man, yet men fear it as if they knew that it is the greatest of evils.”  – Plato.

I am not fearful of death – of my death – but I get too much scared when someone dies – someone close. I am fearful of the fear and silence that it propels in the  surroundings. I am fearful of the all the other scared faces around me, of the mournful smell of the air. I am fearful of the darkness with which it overshadows the brightest sunlight. The tears of all the disturbed souls that soon would flood the universe haunt me. The  swelled-dry-lips and red-wet-eyes of all who are suffering the loss weaken me. How cruel is the fact –  that this astronomically huge span of life would be destroyed in a blink of an eye and would leave everyone behind in pain -it tortures me a lot. Death, by far the most stupid and preposterous creation of God is the greatest fear of all. 

A couple of days back I lost someone. A person who was not there with me for the most of the time of my life, but yes there was this noble and magnanimous image of that lady which was alive in my mind,  my heart and my thoughts for so long – it was all shattered. The image which was just so alive and so vibrant – talking to me smilingly – was all missing in the very next moment.  It was like a black-hole of silence and fear and pain created somewhere in my heart and ’twas drowning me in. I was scared and was never scared this much. I cried a lot – alone – many a times – but that was even drowning me more. I was crying and thinking about the condition of those who were closely related to her, those who had spent their entire life with her.  How pitiful their conditions would be – unimaginable ! And then I was cursing God for  he was the only one who scripted all this drama and was watching it being enacted. Preposterous.

I do not want to leave people behind me tormented and crying in pain. I now know how it feels. But I can not help it. I know they will cry when I am gone, miss me more than ever. But why? Why God has created such nonsense circle of Life and Death? Why on one hand does he put so much strength and positivity in Life and then despise his own creation and spread fear with Death? And why does he not bestow the humans with the power to overcome that fear? Why does he let them suffer? These unanswerable questions will haunt everyone – everyone who has felt death so closely. One can not escape this only irrefutable reality. But then why is this truth so untrue that no one want to face it, to believe it? Death is  evil and the God himself created this evil.

Rest in peace the pious soul of the lovely and beautiful old lady; you shall always be there in my heart.