it's common knowledge to use Runtime.getRuntime().exec(command) to execute any Unix command or Windows command in a Java application. However, when you try to include the pipe '|' or redirect '>' in the command to alter any output pattern, most of the time the Java will not interpret your command as expected which will turn out to be an error finally. For example, when I tried to run ffmpeg command to encode any video and would like to capture those outputs into a log file, an error of "Unable to find a suitable output format for '>'" will appear.
In order to make Java "understand" out purpose, you cannot directly insert the usual command into the exec() parameter. There is a workaround which will solve the issue.
Construct an array:
String[] commands = {
"/bin/sh",
"-c",
YOUR REAL COMMAND HERE
}
and pass the commands as argument to the Runtime.getRuntime().exec(commands). In this way, the Java environment will make a sh (YOU COULD USE BASH) environment to execute your command, which will take the pipe and redirect into consideration.
Monday, July 12, 2010
Thursday, July 8, 2010
Seven Java EE Performance Problems
1) Slow-running applications
2) Applications that degrade over tim
3) Slow memory leaks that gradually degrade performance
4) Huge memory leaks that crash the application server
5) Periodic CPU spikes and application freezes
6) Applications that behave significantly differently under a heavy load under normal usage patterns
7) Problems or anomalies that occur in production but cannot be reproduced in a test environment
2) Applications that degrade over tim
3) Slow memory leaks that gradually degrade performance
4) Huge memory leaks that crash the application server
5) Periodic CPU spikes and application freezes
6) Applications that behave significantly differently under a heavy load under normal usage patterns
7) Problems or anomalies that occur in production but cannot be reproduced in a test environment
Wednesday, July 7, 2010
Long IP Address
Most of the time, people will think of an IP adress as a String. Especially in Java, most of the time, developers will deal with IP address either with URL or String class. However, representing an IP adress with String has several disadvantages. First a String usually takes more memory comparing to the "same" value int or long, and it will be difficult to compare with other IP addresses with a String. And more importantly, it will not be possible for people to easily determine whether an IP address is in the range of another two IP addresses.
Since IP addresses (IPv4) are composed by four integers ranging from 0 - 255, it will be obviously easy to convert an String IP address to an numerical form which can also uniquely represent the IP address. That's how Long IP address emerges.
A simple method to convert the String IP address to a Long IP address (A.B.C.D) would be:
256*256*256*A + 256*256*B + 256*C + D
Using Long IP address will be helpful in certain scenarios, one is when you dealing with IP-Location mapping in Google App Engine. A very popular data called GeoIP created by MaxMind is heavily used in a lot of different projects. However, when parsing the IP addresses, what they have done in the Java library is first transform the IP String into an InetAddress, and then using getAddress() to get its byte[], and finally get the Long value. There will be no problem when you using this library in other platforms. But in Google App Engine, things will got stuck because of the InetAddress is on the black-list of GAE, which means you will not be able to play with this class. The workaround here would be using the converting method above, you can directly get the Long value which is what they have calculated all the way along.
There might be some other places Long IP addresses is useful, especially when dealing with range query of IP addresses.
Since IP addresses (IPv4) are composed by four integers ranging from 0 - 255, it will be obviously easy to convert an String IP address to an numerical form which can also uniquely represent the IP address. That's how Long IP address emerges.
A simple method to convert the String IP address to a Long IP address (A.B.C.D) would be:
256*256*256*A + 256*256*B + 256*C + D
Using Long IP address will be helpful in certain scenarios, one is when you dealing with IP-Location mapping in Google App Engine. A very popular data called GeoIP created by MaxMind is heavily used in a lot of different projects. However, when parsing the IP addresses, what they have done in the Java library is first transform the IP String into an InetAddress, and then using getAddress() to get its byte[], and finally get the Long value. There will be no problem when you using this library in other platforms. But in Google App Engine, things will got stuck because of the InetAddress is on the black-list of GAE, which means you will not be able to play with this class. The workaround here would be using the converting method above, you can directly get the Long value which is what they have calculated all the way along.
There might be some other places Long IP addresses is useful, especially when dealing with range query of IP addresses.
Tuesday, June 15, 2010
tmpreaper and ctime, atime, and mtime in Ubuntu
There is a package in Ubuntu can be used to clean directories with those files older than a certain period of time. Before we get into that, let's first clarify three terms related to times in Ubuntu: ctime, atime and mtime.
ctime is the creation time of the file. Say I created this file at Wed Jun 16, 9:45:15, 2010, then this time spot will be exactly the ctime of this file.
atime is the access time of the file. Displaying the file contents or executing the file script will update the atime.
mtime is the modify(ication) time which will be updated when the actual content of the file is modified.
Back to the tmpreaper command, since it is not default installed into the Ubuntu, you have to sudo apt-get install to get the latest version of tmpreaper.
It is a simple command tmpreaper TIME-FORMAT DIRS to invoke the function to do the clean job for you.
TIME-FORMAT is a parameter that specifies the duration of the file which has not been accessed. By default, the time here is about atime. So even if you modify the content in a later stage but does not access the file, the file might still be deleted. Of course, you can enforce the command to run in terms of mtime which you have to append --mtime to the original command.
While the DIRS is the directory you would like to invoke this function, such as /tmp. Never try to do such a thing on the root directory or you may encounter a disaster.
If you have to manually run the command every time, then there is no sense to use this. While the power strengthens with combining another tool CronTab.
CronTab is used to create cron job to run specific script in a period of time. In order to run the cron job, all you need to do is write a script which include the command we have talked previously, then edit the configuration file of CronTab, then the scripts will run as you required in the background.
To edit the configuration file, simply run sudo crontab -e, add an entry in to the file. The format of the file is m h dom mon dow command, the first five sections are divided by space, and you can use asterisk to specify anytime like a wildcard.
Fox example, * * * * * /XXX.bash will run every minute. More usage can be seen from the documentations.
ctime is the creation time of the file. Say I created this file at Wed Jun 16, 9:45:15, 2010, then this time spot will be exactly the ctime of this file.
atime is the access time of the file. Displaying the file contents or executing the file script will update the atime.
mtime is the modify(ication) time which will be updated when the actual content of the file is modified.
Back to the tmpreaper command, since it is not default installed into the Ubuntu, you have to sudo apt-get install to get the latest version of tmpreaper.
It is a simple command tmpreaper TIME-FORMAT DIRS to invoke the function to do the clean job for you.
TIME-FORMAT is a parameter that specifies the duration of the file which has not been accessed. By default, the time here is about atime. So even if you modify the content in a later stage but does not access the file, the file might still be deleted. Of course, you can enforce the command to run in terms of mtime which you have to append --mtime to the original command.
While the DIRS is the directory you would like to invoke this function, such as /tmp. Never try to do such a thing on the root directory or you may encounter a disaster.
If you have to manually run the command every time, then there is no sense to use this. While the power strengthens with combining another tool CronTab.
CronTab is used to create cron job to run specific script in a period of time. In order to run the cron job, all you need to do is write a script which include the command we have talked previously, then edit the configuration file of CronTab, then the scripts will run as you required in the background.
To edit the configuration file, simply run sudo crontab -e, add an entry in to the file. The format of the file is m h dom mon dow command, the first five sections are divided by space, and you can use asterisk to specify anytime like a wildcard.
Fox example, * * * * * /XXX.bash will run every minute. More usage can be seen from the documentations.
Sunday, June 13, 2010
Handling Azure Large File Upload
In Azure storage, files smaller than 64MB can be directly stored as a single blob into the storage. However, when you want to store a file larger than 64MB, things will become a little bit complicated. The way to accomplish this task is to use the block list service.
Block, unlike blob, is a small unit of file which can be aggregated as a list to form a large file, with each of the small chunk to have a limit of 4MB. For example, say if you have a 100MB file which you want to store into Azure, you have to manually split the files into at least 25 pieces, and then using the put block & put block list operation to upload all the 25 items. More details are listed below:
1) Split large files: this can be done in various ways, via existing tools or write your simple code. Pay attention to write down those file names and make them in the sequence you split them.
2) Put Block: Each of the pieces created last step is called a block, and the second will upload each block one by one into the storage via Put Block operation. The basic process is no difference with other methods, however, one thing need to pay attention is the blockid is a required parameter and all blockids of the blocks must be the same size. In our example, you can have a Base64 blockid with arbitrary length less than 64, but you have to enforce all of the 25 items to have the same length. If not, a 400 exception, or The specified blob or block content is invalid error message will be returned.
3) Put Block List: The last but not the least step is to notify the server that all pieces are uploaded and now it's your job to combine them altogether.
After the three steps, you will be able to upload any size files into the Azure storage.
Block, unlike blob, is a small unit of file which can be aggregated as a list to form a large file, with each of the small chunk to have a limit of 4MB. For example, say if you have a 100MB file which you want to store into Azure, you have to manually split the files into at least 25 pieces, and then using the put block & put block list operation to upload all the 25 items. More details are listed below:
1) Split large files: this can be done in various ways, via existing tools or write your simple code. Pay attention to write down those file names and make them in the sequence you split them.
2) Put Block: Each of the pieces created last step is called a block, and the second will upload each block one by one into the storage via Put Block operation. The basic process is no difference with other methods, however, one thing need to pay attention is the blockid is a required parameter and all blockids of the blocks must be the same size. In our example, you can have a Base64 blockid with arbitrary length less than 64, but you have to enforce all of the 25 items to have the same length. If not, a 400 exception, or The specified blob or block content is invalid error message will be returned.
3) Put Block List: The last but not the least step is to notify the server that all pieces are uploaded and now it's your job to combine them altogether.
After the three steps, you will be able to upload any size files into the Azure storage.
Tuesday, May 25, 2010
How to tell the version of your Mac OS
Step 1: Click the apple icon in the upper left corner of the screen
Step 2: Choose About this Mac
Step 3: The label right below the Apple indicates the version of this Mac OS, e.g. Version 10.6.2
Step 4: Right to the Processor label is the version of your CPU, from which we can tell the bits of the CPU
Here is a table to clarify the relationships:
Ref : http://support.apple.com/kb/ht3696
Step 2: Choose About this Mac
Step 3: The label right below the Apple indicates the version of this Mac OS, e.g. Version 10.6.2
Step 4: Right to the Processor label is the version of your CPU, from which we can tell the bits of the CPU
Here is a table to clarify the relationships:
| Processor Name | 32- or 64-bit |
| Intel Core Solo | 32 bit |
| Intel Core Duo | 32 bit |
| Intel Core 2 Duo | 64 bit |
| Intel Quad-Core Xeon | 64 bit |
| Dual-Core Intel Xeon | 64 bit |
| Quad-Core Intel Xeon | 64 bit |
Ref : http://support.apple.com/kb/ht3696
Saturday, May 15, 2010
Redirect vs Forward
In the context of web programming, a lot of ambiguity existed between Redirect and Forward. Here is a short summary of the differences between these two terms.
1) Forward can only be direct to an internal page, however Redirect can be used both to an internal page and external page.
2) Forward is much faster than Redirect
3) With Forward the browser is unaware of what happens, and the URL address remains the original link, while Redirect will initiate a new request that the browser will update its address to the new link.
4) As a result of the browser awareness, the refresh function will be failed in the Forward since the URL has not changed, but the Redirect will be all the same.
5) Still with the same reason, with those operations have side-effect, say update the status of database, a Redirect should be used to avoid the refresh Forward to generate any duplicate operations.
1) Forward can only be direct to an internal page, however Redirect can be used both to an internal page and external page.
2) Forward is much faster than Redirect
3) With Forward the browser is unaware of what happens, and the URL address remains the original link, while Redirect will initiate a new request that the browser will update its address to the new link.
4) As a result of the browser awareness, the refresh function will be failed in the Forward since the URL has not changed, but the Redirect will be all the same.
5) Still with the same reason, with those operations have side-effect, say update the status of database, a Redirect should be used to avoid the refresh Forward to generate any duplicate operations.
Subscribe to:
Posts (Atom)