Different ways of calling a perl script from another

August 27, 2013

Perl has a number of possibilites to execute other scripts / commands:

  • backticks / qx{} When you want to read all the output at once after the program has terminated
  • exec When you wan’t to continue your process as another program — never returns if succesfull
  • system When you are only interested in the success or failure of the command
  • open When you want to pipe information to or from the command
  • do and require Execute another Perl script here. Similar to C’s #include
  • There are modules to do a three-way open so that you have access to STDINSTDOUT and STDERRof the program you executed. See the apropriate parts of perlipc for advanced information.
Advertisements

CAP Theorem

January 25, 2013

The CAP theorem or the Brewer’s theorem states that a distributed system can only guarantee two out of the three :

  • Consistency – results of earlier writes on a node are read by read operations on the node
  • Availability – a guarantee that every request receives a success/failure response
  • Partition – the system continues to operate despite a failure of a subsystem or some message loss

This is similar to the three constraints of project management where you can choose two constraints and the third gets decided : time, cost and scope.

Henry Robinson has given a shown a very good understanding of the concept on Quora .

An informal proof which helps the intuition from above :

             “The intuition behind this result is as follows: to be consistent, all nodes have to see the same set of updates in the same order. But if the network suffers a partition, updates in one partition might not make it to the other partition before a client reads from the out-of-date partition *after* having read from the up-to-date one. The only thing you can do to cope with this possibility is to stop serving requests from the out-of-date partition, but then the service is no longer 100% available. “

                “Since, until there is a failure, it is relatively easy to guarantee availability and consistency in well-behaved executions, so some systems gracefully degrade their consistency or availability guarantees only at the point of failure. “

After gaining this understanding, some notable points from InfoQ article on CAP

  • CAP prohibits only a tiny part of the design space: perfect availability and consistency in the presence of partitions, which are rare.
  • ACID properties focuss on consistency
  • BASE properties focus on availability

Relevant links

ACID is a set of properties that apply specifically to database transactions
The CAP theorem is a set of basic requirements that describe any distributed system (not just storage/database systems).
http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed


Learning from Facebook

April 6, 2012

A recent article on the Facebook release engineering team offers a lot to learn for young engineers. At times, I have felt that many developers are blinded about the processes followed by IT and operations teams after development and testing phase is over. This article helps in overcoming more of those blind spots and goes towards the DevOps movement.

Learnings to be taken :

1) Facebook uses PHP for most of its development as it reduces the development and ramping up time. This allows organic agility to the company to move fast. However, scripting languages tend to be less efficient with CPU cycles usage.  To bridge this gap and to take the advantages of compiled languages, facebook has an automated code converter from PHP to C++ which they call HipHop. Read More about HipHop.

2) Facebook’s entire code can be compiled to a binary executable, 1.5 GB in size. Hence, during a release this entire binary is pushed to all the facebook servers.

3) Testing and Release : Facebook employs automated tests and regression to find bugs. The release takes place in two phases. In the first phase the code is released to some of the servers and exposed to a fraction of the user base.

Excerpt from the article :

” Facebook is designed to be stateless and distributed, in the sense that the user’s session isn’t tied to any particular server. Any given page request can be handled by any of the servers in Facebook’s infrastructure.

That approach offers a lot of resilience. When Facebook performs an update, it doesn’t have to worry about serializing and migrating the state of user sessions. ”

Hence, a deployment doesnt require any downtime of the prod servers.

4) Feedback Analytics : Facebook uses different metrics and compares historical data to make sure everything is working fine after a release. What was interesting to learn was that they also use twitter feeds to judge positive and negative sentiments of the users after a release.

Overall an interesting article on software engineering and provides a peek into the working of facebook.


Version Control

March 22, 2012

The Subclipse project integrates support for the Subversion version-control system with Eclipse’

a) Adding a new project to the subversion repository (Team->Share Project)

b) Updating a project (Team->Update)

c) Adding a new file :  If you add a file to your project, it’s not automatically part of version control — you need to specifically add it to the repository(Team->Add to version control).

d) Deleting a file : Subclipse flags the file for deletion automatically and removes it from your project.

e) Ignoring files : Right-click the file or directory you want to exclude from version control, then choose Team > Add to svn:ignore to display the Add to svn:ignore dialog.

f) Commiting your changes : Be sure to update your project (see “Updating a project”) before attempting to commit your changes. Right-click the project and choose Team > Commit from the menu.

g) Main difference between branch, tag and trank given in stack overflow.

http://stackoverflow.com/questions/16142/what-do-branch-tag-and-trunk-really-mean

Reference :

1) http://www.ibm.com/developerworks/opensource/library/os-ecl-subversion/