Summary and Review on Open NF

In “Improving the Safety, Scalability, and Efficiency of Network Function State Transfers,” Gember and Akella describe ways to overcome the shortfalls in their OpenNF system. OpenNF enables the moving network-functions (NFs; Firewalls, Intrusion Detection Systems, etc…) from one physical piece of equipment to another one within a network. This is a problem that industry professionals are concerned with when equipment needs to be replaced or moved to accommodate more bandwidth.

Problems with OpenNF

There are two important problems with their previously described system in “OpenNF: Enabling Innovation in Network Function Control.” The problems are Buffer Overflow and Latency. Both stem from the fact that Open NF depends on a controller which is responsible for buffering packets that need to be processed at the new location. If these packets aren’t processed at the new location then important state information about the system could be lost. The consequence of such a loss will be a less secure system which an adversary could likely exploit for sinister purposes.

Buffer Overflow

Suppose we have three entities, A, B, and C for the old location, new location, and controller respectively. The controller has a limited buffer space to store incoming packets from the old location, A. Previously packets were being processed at A, but while changing the architecture of our system the packets A receives are forwarded to B and not processed. If C’s buffer runs out of space before B is ready to take over for A then any packets that A sends to C are lost.

To overcome this fault the authors offer a stronger solution. Simply process packets at A while changing the architecture, and forward the packets to C only if they will affect the state of A and B. If C’s buffer becomes full then B is updated with a snap-shot of A’s state.

The authors prove that sending a snapshot of A to B ensures no packets are lost in the face of buffer overflow, but they don’t include details on the cost replicating A’s state to B. If such an action was cost free then state could be continuously replicated at B and there would be no need for a packet buffer at all.

Latency

Latency is a measure of the time that it takes for a message to be received. Ensuring that updates from A to B are loss-free is costly when considering this metric. In the original description of OpenNF a message couldn’t be received until it was released from the buffer at C. The authors solution to the buffer overflow problem also helps mitigate the latency problem, as any packet that is buffered at C is first processed at A. However, the time to complete the migration from A to B is still a problem.

To reduce the migration time the authors propose a peer-to-peer migration in which packets are forwarded directly from A to B rather than passing through a controller, C. With a peer-to-peer transfer the latency for time to complete a migration stays relatively constant with respect to the number of flows moved from A to B whereas with a controller-directed transfer the time to complete the migration increased linearly.

Review

The analysis of the buffer-overflow solution should be looked into more closely. Their proof guarantees loss free transfer via re-exporting state for some flow from A to B if an overflow occurs. Will this still work if there is so much state to transfer that the overflow occurs multiple times in quick succession? Is there a point at which the the cost of copying of the state from A to B too great to honor that loss-free guarantee?

The authors seem to have provided non-overlapping solutions to their problems. Perhaps they intend for an approach to be chosen based on case-specific needs. For instance if controller constraints are not a problem then the safe buffer approach is enticing, but if you need to scale the system to accommodate many controllers then the P2P solution looks more helpful. The authors did not mention any trade-offs between these two approaches in their conclusion.

Conclusion

Their papers together offer a great deal of insight into the problem of state transfer in a networked environment. Their solutions are novel and they have considered a great amount of previous work. I think that although they considered their solutions as a way to enable state transfer of network functions their methods might find relative applications in other areas of distributed systems. Their papers are highly readable, and they deserve a lot of credit for that as well.

 

Advertisements

Deploying a Django Application to AWS using Elastic Beanstalk (OSX)

In this blog I show you how to deploy your Django App to Amazon Web Services (AWS) using Elastic Beanstalk (EB). For this tutorial I am using a MacBook Pro. There are minor differences Windows users.

This guide is brief and intended for small apps. I do not cover how to configure a custom database for your server in this tutorial. For details on setting up a PostgresSQL or MySQL backend see the Django Documentation.

Note on notation: Lines which can be directly typed into the command prompt or a text editor have no leading character through this tutorial. Lines which are given as terminal output are preceded with ‘>’.

For this task you will need:

  • An AWS Account
  • A Django Project
  • A python package installer (pip or brew)

Install awsebcli

First install ‘awsebcli’ via the terminal

brew update
brew install awsebcli

Once this is finished check to see that you have an Elastic Beanstalk Client installed,

eb --version
> EB CLI 3.12.0 (Python 2.7.1)

If you see a message which says that the command eb is not found, attempt a different install method. See Amazon’s documentation for more information.

Navigate to the folder containing your Django project. If you don’t already have a project one will be assigned to you now.

git clone "https://github.com/mdamien/django-tutorial"
cd django-tutorial

Create an IAM User

You will want to have a user configured in IAM at this point with full access to Elastic Beanstalk. If you do not have such a user create one now.

To create an EB user select IAM from the AWS management console. Select User from the left panel, then select Add User at the top of the screen. Next give them a username and select pragmatic access. Now press Next: Permissions.

On the next screen pressed the button labeled “Create Group”. Name this group EB, and then assigned it the policy “AWSElasticBeanstalkFullAccess”. You can type “beanstalk” in the search filed to find this policy quickly. After you’ve selected this policy press Create Group.

Now you can click Next: Review at the bottom of the page. Finally, press Create User.

IMPORTANT: You should now be looking at your user’s security key and security key ID. You will want to keep this information safe somewhere, as we will be using it very soon. Just keep this page open for now.

Initialize Elastic Beanstalk

From within the top-level directory of your django project folder, initialize your Elastic Beanstalk Client.

eb init

Choose the region which is closest to your users. Press enter.

If you are asked to choose an application to use, choose create new.

You will be prompted to give your application a name, and then eb will ask if you are using Python. Say yes, then select Python 3.6.

Now you’ll be asked if you want to use AWS CodeCommit. Select No unless you are feeling adventurous.

Finally you will be asked if you want to set up SSH for you instances. Say Yes.

Now you will be prompted for your Security Key and ID. Paste these from the user you created in the previous section of this tutorial.

Create an Environment

You’ve just created place within your project directory to store information for your Elastic Beanstalk instance. To push your application to an AWS server you will have to create an environment in Elastic Beanstalk for your application to live.

eb create

When you run this command you will need to specify an environment name, just use the name of your application. You will also be asked to give a DNS CNAME prefix. This needs to be unique within AWS. It’s probably best to use the default name that Amazon suggests for you here.

For load balancer type select application.

Now EB will create an environment for your application. This will take a few minutes.

Prime your app for Elastic Beanstalk

Before you app is ready to run on EB you’ll have to give EB the path to your WSGI file and static files. You also have to let EB know that it needs to install Django.

Specify paths for WSGI and Static Files

Before your code is executable on Elastic Beanstalk you will need to let your EB environment know where WSGI file lives. The simplist way to do this is do run

eb config

then scroll down untill you see…

aws:elasticbeanstalk:container:python:
 NumProcesses: '1'
 NumThreads: '15'
 StaticFiles: /static/=static/
 WSGIPath: application.py

If you are using the django-tutorial example for your app, then StaticFiles is at polls/static/polls/ and WSGIPath is at mysite/wsgi.py.

Change the eb config file to reflect the location of your static files and WSGI path, then exit the file. Now Elastic Beanstalk will automatically push the changes you made in this file to your EB environment on the cloud.

Install Django

Create a top-level file in your project called requirements.txt and in that file just write Django.

If you are using the django-tutorial app you can execute this command from the top level directory of mysite,

echo "Django==11.1.0" > requirements.txt

To deploy this change to EB you will need to commit the change in git. Elastic Beanstalk will always deploy whatever has been most recently committed.

git stage requirements.txt
git commit -m "added requirements.txt"
eb deploy

Open Your App

To open your app just type

eb open

Resolve Any Errors.

When you open your app you might see an Django error message. You will need to add elastic beanstalk to your ALLOWED_HOSTS in settings.py. Just copy the URL from the page opened by eb open.

ALLOWED_HOSTS = ['YOUR_DNS_CNAME_PREFIX.elasticbeanstalk.com']

Now just commit the change in git and deploy your site. If you had any other Django-related errors you can resolve these locally and then deploy the changes once you have a stable local version of your app.

git stage .
git commit -m "resolved Django errors, added EB to ALLOWED_HOSTS"
eb deploy

You’re Done!

After the changes have been deployed refresh the page where your site is at. You should now have your Django app living on AWS’ Elastic Beanstalk. Congratulations!

If you have any questions please comment below, and subscribe to my blog for more!

 

Vacation on Vacation

Hello Reader,

It’s September in Rome. Rain storms are coming and going like rumors in high school. At seven and a half months old my niece is almost standing on her own, and her brother is almost ready to crawl. After three months in Italy I’m preparing myself mentally to fly back home to California.

It surprises me, the passing of time. Last month I was in Palinuro, Italy. Last weekend I was in Cartagena, Spain. Next week I’ll be back in the USA. I’m so grateful to the universe because I have had this opportunity to travel. I’m grateful to my friends and family that have hosted me over here in Europe. I’m also grateful to myself for having decided to take this trip when I saw the opportunity.

Palinuro is such a beautiful place. If Italy is shaped like a boot, Palinuro is a boot within the boot (see the picture below this paragraph). While I was there with the Calitalianos we did some serious relaxing. What a deal! We went to the beach twice a day for two weeks and ate amazing food. One dish that my sister’s mother-in-law, Rosa, made took her two days to prepare. She sun-dried a pan of sliced zucchini to bake with parmesan and tomato sauce (I was told that the sauce was hand-made by someone in Rosa’s town).

img_6156

The most thrilling part of Palinuro was kayaking around the little boot. I was looking for this private boat-access-only beach, but I went around the entire cape looking for it. This took about three hours total. When I got back to the house later my sister told me that I passed the beach in the first thirty minutes of my adventure. So it goes…

Last Friday in Spain, I met up with some friends that I met while studying in Germany two years ago. There was a festival going on in Cartagena to celebrate the history of the region. For me, this meant that we would stay out until 5am drinking at the festival. It also meant that to fit in I would have to dress like a slave of the Roman Empire. The next day we rented a sail boat and sailed around Mar Menor, literally called the Little Sea. Then on Sunday I got back onto a plane at 8am and flew back to Rome.

I had such a blast in Spain. Who wants to go spend a lot of money to go to a big famous city and sit on a bus full of strangers while someone tells you what you ought to know about the city, when you could actually just go to some place like Cartagena and hang out with your friends ’til 5am and wake up the next day and go pee off the front end of a sail boat after a couple of your friends spilt last night’s liquor off the port and starboard side?

Okay maybe that’s a little harsh (and gross). If you want to go and spend a typical vacation abroad there’s nothing wrong with that. In fact, you should! Travel as much as you can. Take every chance that you can get. I think it’s one of the most important things to do in the world. Get out and meet as many people as you can. Exchange ideas! Broaden your surroundings and let the place that you came from and all of the drama or problems there shrink as you fall in love with every new place that you find while traveling.

All and all I’m considering myself lucky that I can do any of this at all. All of the Palinuro pictures are in the last blog, this weeks pictures are all from Cartagena and Mar Menor. Thanks for reading! 🙂

Turn Out the Lights

Let’s talk about tricks of the mind on itself for a moment (It’s going to get dark in here). Have you ever been holding onto a harmful idea while knowing the the idea will only hurt you. Smokers might relate to this. You can’t quit the habit but you know people who’ve been taken to death by causes directly linked to tobacco use like heart disease or cancer. I smoked for six years before I took leave from that bus ride, whose destination was likely tragic.

I think that it’s easy to ignore the future. We can say, “Life is about the journey, not the destination.” This saying just as well say to me, “The future doesn’t exist. What is now is forever, and the choices you make now will carry you only to a further-actualized image of yourself.” The journey is endless. I don’t know why I think that what is now is forever. It’s just an intuition that carries me.

You might be thinking, “Okay Matt, you’re off the rocker. You lost me at tricks of the mind. What are you getting at here?” Let’s say your roommates have an argument, and it’s none of your business but it’s loud and you have to witness. You feel a moral obligation to speak in ones defense. Do you speak out when you favor one of your roommates, or think that the other is being unfair in their argument? If it’s really none of your business then why do you feel so guilty in your silence… You’re idea of self hinges on how you do or don’t respond to their argument.

I have been living in a studio apartment with my sister, her fiancé and their infant twins for just over two months now.  I’ve questioned my position in this house a few times when it’s been hard to get along. Mostly it’s been smooth sailing though- Eating great Italian food and quality time with this side of my family. Other times it’s not been so easy.

I’m holding onto the idea of gratitude though, especially when we don’t all get along. I’m grateful for my place here. My mind might trick me into feeling guilt or sorrow but really I’ve got it great. I’m still on vacation and my only real obligation is to make the most of it. Here are some pictures from the last couple weeks.

Blog Time, No See

Hello there, reader!

I’ve come back to write you about a few of the recent developments in my life. I’m helping OrientDB rewrite some of their Database Management System (DBMS) documentation. I’m leaving Rome today to visit Salerno and the Palinuro, Italy, with the Janel, Massimo, and the Twins; Leo and Bea (I asked Janel for a shorter way to say all of those names and she suggested ‘ The Calitalianos’). Also, I’ve got a definite date for flying back the the US! In other Calitaliano news, I just put Bea down for a nap.

Putting her to sleep is a big accomplishment in this house, because usually she’ll only sleep while her mother is breastfeeding her. You can earn a lot of points for putting her down without the magic of the milk. I’m celebrating this afternoon with a beer and this blog. You’ve gotta reward yourself for your achievements, it’s good for self efficacy!

On the note of self efficacy I want to talk about something that’s been on my mind. The reason I’m taking this year off after my Bachelors degree, and living with the Calitalianos is so that I could have time to consider what I want from my life after my undergraduate achievements. I’ve decided to pursue graduate school in California. Also as far as graduate school applications go, I will only apply for Ph.D programs in Mathematics. This is a bit of a leap because I did my B.S. in Computer Science- not the same subject.

It’s clear to me that math is the thing that’s calling me though. The books I lean towards when I’m at the book store are books about math or it’s history and players. When I’m bored I go to math community forms online and see what problems people are puzzling over. The classes that I felt the most confident is raising questions in were often math. I’m confident that this is where I’m suppose to be putting my attention.

I  often hear people tell me, “But Matt, all the money… all the jobs… all the important things are in Computer Science.” It’s somewhat true, there are a lot of computer science jobs and most pay well, but I’m less sure that that’s where my calling is. Another way I’m seeing things right now is… If I fail to get into the Ph.D program I’m after, then I will have given myself a comfortable cushion to fall on. I’m not any less qualified for the same career paths that are available to me right now should I try and fail to pursue myself as a mathematician.

So there you go reader, there is your peek into my thoughts. Next the we talk I hope to be showing you pictures of the warm Mediterranean beaches of Palinuro, Italy. Thanks for reading! Here are some recent pictures I’ve taken.

Paint’s Peeling

Hello again Dear Reader,

Since last we met somethings have changed, and others are still the same. I’m still living in Rome with my sister, her fiancé Massimo, and their children- the twins Leo and Bea. I’m still slowly making my way through a book on Real Analysis (I’m no longer unaware of what a Cauchy sequence is (I might blame math for making me think it’s okay to write statements in double negatives) ). I’m also no longer jobless!

Through a friend of Janel’s I’ve started preparing for an internship with OrientDB here in Rome. This is way-super-exciting to me because I’ve been out of school for only a few months and have dropped into a foreign country and already I’ve got people outside of academia (in the Real World) interested in my skills as a coder! I’m not trying to brag or anything, but I am really excited to work with the creators of OrientDB. Right now I’m halfway through an online course at udemy.com that introduces graph-databases and how to use OrientDB’s software.

Also in the way of new news, I’ve taken my first solo-two-night-excision into a place unknown! I went to Bologna a few days ago, and it was fantastic. I met some nice people and biked around the town almost the entire time. Biking around in Bologna is unlike biking anywhere in the US. The streets are narrower, the buildings are older, and the drives are nuttier. I just feel blessed that I got to take such a trip.

I think the funniest part of the trip came after I biked up to Sanctuary of the Madonna di San Luca. I left the sanctuary famished by the bike ride. I stopped at the first Pizzaria-Kabab I found because I knew these generally have cheap and filling food. The three guys working there were all from Pakistan and all obviously interested in me, by the way they asked me a ton of questions (Where are you from, how long are you here, do you like this city, why this city, … ). I better say now too, that they had a genuinely friendly way about asking me things.

I feel delusional for having thought this, but in the back of my head was… “what if these three guys are terrorists AND they want to know so much about me to see if I’m right for abducting!”  There – I said it. And it sounds so silly and improbable to me now.

In defense of this possibility though, I reflected all of their questions and friendliness back at them. Surely if these guys are in it with the terrorists, then I might be able to charm them into continuing their friendliness feigned until it would become sincere! Now this sounds just as silly to me too. If they really were terrorists then they would have had a deep-rooted hatred for the western world. Why would they open a restaurant in a European college town and make such friendly, idle conversation with an American if this was so? It doesn’t matter whether my delusional (likely conditioned) fear of mine was true or not though because what happened here was hilarious!

I learned that the oldest of them had moved to the city eight years ago and opened shop to sell kebabs and pizza. The youngest of the three guys didn’t talk much. When I asked what food on their menu didn’t have meat, (“Che comida senza carne”) I was told that I was handsome. I still don’t know if this was a genuine flirting advance or if it was the older man’s way of voicing his supposition: A man who doesn’t want to eat meat must be gay. It didn’t matter to me though. I just accepted the complement and kept putting together my order, trying to do it in Italian as best I could.

I soon learned that all three guys spoke English. One was giving me Italian lessons by naming all the ingredients in the food that I ordered. He even invited me to the kitchen to see where they were prepared the food. The paranoid/delusional part of me screamed not to go into the kitchen but by now I had dismissed the notion that these men were dangerous as comical. They were so nice!

I’m sure you’ll take what you will from that story, but if I can let myself get a little preachy on my own blog I’ll say that the first thought that you have about a person (especially a fearful thought) is generally unfounded. I think it’s important to not let your thought be who you are. You can have a thought about a person or place or other thing but it’s okay to simply acknowledge it and then leave it there. If I clung to every first thought I have had I’m not sure where I would be, but I know I wouldn’t be where I am.

753 words before this paragraph. I should cut my shpeele here and drop some pictures now. Thanks for reading! See ya next whenever I write on here again ;D

Here I am, just as promised to myself by me one week ago (Thanks Chloe for reminding me)! This week in Rome has been a challenge and a triumph. I babysat my five month old niece and nephew alone two days in a row, and now I realize that babies are self-destructive and cautionless human beings. I have a new respect for parents everywhere, because watching babies is hard. And I only did it for about three hours each day!
Also this week, I’ve started a new photo project called Fountains of Rome. I’ll share all of the pictures at the bottom of the page here. I think the name says it all but I’ll tell you now, they are all drinking fountains that I’ve captured for this project.
That’s all folks!

This slideshow requires JavaScript.