What is MemCacheD?
Memcached is a general-purpose distributed
memory caching system that was originally developed by Danga Interactive for Live Journal, but is now used by many other
sites.
It is often
used to speed up dynamic database-driven websites by caching data and objects
in RAM to reduce the number of times
an external data source (such as a database or API) must be read. Memcached
runs on Unix, Linux, Windows and MacOSX and is distributed under a permissive free software license.
Memcached's APIs provide a giant hash table distributed across multiple machines.
When the table is full, subsequent inserts cause older data to be purged in
least recently used (LRU) order. Applications using Memcached typically
layer requests and additions into RAM before falling back on a slower backing
store, such as a database.
How Does It Work ?
Memcached is similar to a hash table. A hash table is implemented as an array of buckets. Each bucket (array element) contains a list of nodes, with each node containing [key, value]. This list later is searched to find the node containing the right key. Most hashes start small and dynamically resize over time as the lists of the buckets get too long. A request to get/set a key with a value requires that the key be run through a hash function. A hash function is a one-way function mapping a key (be it numeric or string) to some number that is going to be the bucket number. Once the bucket number has been calculated, the list of nodes for that bucket is searched, looking for the node with the given key. If it's not found, a new one can be added to the list.
Memcached represents the user with a dictionary interface but instead of a one way hash function it represents a two way hash. The first kind of hashing is done in the client library where the client decides to which server to send the request to. It does so by hashing the key into a list of virtual buckets, with each bucket representing a Memcached server. Each Memcached server represents a typical hash table. Each Memcached server instance is independent, unaware of the presence or status of other servers. All these servers are unified together by the client library. If a server fails, the clients can be configured to route around the dead machine or machines and use the remaining active servers.
The system is used by sites including
YouTube, Facebook, and Twitter.
Architecture:
The system
uses a client–server architecture. The servers
maintain a key–value associative array; the clients populate this
array and query it. Keys are up to 250 bytes long and values can be at most 1 megabyte
in size.
Clients use
client side libraries to contact the servers which, by default, expose their
service at port 11211. Each client knows all servers;
the servers do not communicate with each other. If a client wishes to set or
read the value corresponding to a certain key, the client's library first
computes a hash of the key to determine the server that will be used. Then it
contacts that server. The server will compute a second hash of the key to
determine where to store or read the corresponding value.
The servers
keep the values in RAM; if a server runs out of RAM, it discards the oldest
values. Therefore, clients must treat Memcached as a transitory cache; they
cannot assume that data stored in Memcached is still there when they need it. MemcacheDB and Membase
provide persistent storage while maintaining memcached protocol compatibility.
If all
client libraries use the same hashing algorithm to determine servers, then
clients can read each other's cached data; this is obviously desirable.
A typical
deployment will have several servers and many clients. However, it is possible
to use Memcached on a single computer, acting simultaneously as client and
server.
Example
pseudo code:
Note
that all functions described on this page are pseudo-code only.
Memcached calls and programming languages may vary based on the API used.
Converting
database or object creation queries to use Memcached is simple. Typically, when
using straight database queries, example code would be as follows:
declare
function get_data(userid as xs:int) {
let $data:
= fn:collection(“userData”)/users/user[@userId=userid];
return data;
};
After conversion to Memcached, the
same call might look like the following
declare function get_data(userid as xs:int) {
/* first try the cache */
let $data:= memcached_fetch("user:" + userid);
if (!data) {
data
= fn:collection(“userData”)/users/user[@userId=userid];
return data;
}
};
The
client would first check whether a Memcached value with the unique key "userid"
exists, where userid is some number. If the result does not exist, it would
select from the database as usual, and set the unique key using the Memcached
API add function call.
However,
if only this API call were modified, the server would end up fetching incorrect
data following any database update actions: the Memcached "view" of
the data would become out of date. Therefore, in addition to creating an
"add" call, an update call would also be needed using the Memcached
set function.
declare function update_data(userid as xs:int, updateString as xs:string) {
/* first update db */
let $result: = execute(updateString);
if (result) {
/* database update successful : fetch data to be stored in cache */
let $data: = fn:collection(“userData”)/users/user[@userId=userid];
/* then store in cache until next get */
let $expireTime:=0;
memcached_set(userid, data,expireTime);
}
};
This
call would update the currently cached data to match the new data in the
database, assuming the database query succeeds. An alternative approach would
be to invalidate the cache with the Memcached delete function, so that
subsequent fetches result in a cache miss. Similar action would need to be
taken when database records were deleted, to maintain either a correct or
incomplete cache.
Installing and Using Mem-cached with
JAVA:
Note:
Original: for Linux.
Win32 binary: for windows cmd prompt based.
Dependencies: .net
framework (for windows only, if using GUI version)
Memcached java client jar download: http://code.google.com/p/spymemcached/downloads/detail?name=memcached-2.5.jar&can=2&q=
JAVADOC Memcached:
Reference:
After downloading all the stuffs, let’s learn how to use it.
Step1- Install
Memcached GUI server.
Step2- Run
Memcached [See first look below.]
Step3- configure
memcached server.
Ø Click on “Add Server” button.
Ø Provide “Server name”, Username
& password of your operating
system, if you want to use.
Ø Remember server name should be ‘localhost’ or IP.
Ø Click on “Add instance” button. [You will see this window]
Ø Provide “Instance Name”, “Port no.
(if want any specific, otherwise leave it as it is default.)”.
Ø Provide “IP” if want to use some specific IP.
Ø Change memory settings, if required.
Ø Add maximum no. of connection to allow on the
server.
Ø Now click on”Apply button”.
Ø It will ask to start your instance of
server, and then click on “Yes”.
Ø Instance will be created
successfully.
Ø We can add n number of instances as required;
can be run on different-2 IP and Ports.
Ø Step3- Now we
can see the status of our instance.
Ø Step4- Now
it’s time to use it.
1- Now add the “memcached-2.5.jar” in class path.
JAVA requirements:
Ø You can use MemcachedClient.set () to store
a simple string or a complex object.
When you store a complex object, MemcachedClient
will first serialize the object and
then store it. As a result, every object
that you store in Memcached must be serializable, and the key must also be
a string. The set () method returns
an object of Future.
When we call the set method, it is executed asynchronously, so the control
moves to the next line without waiting for a response from the Memcached
server. If you needed to know the result of the set operation, then you would
call setResult.get () on the Future
object instance.
Ø If you have stored complex objects
then the client will first retrieve the value and then deserialize it and return the object.
JAVA CODE:
package memcached;
import
java.io.Serializable;
public class ObjectModel
implements Serializable{
private static final long serialVersionUID = 1L;
String name;
String address;
int age;
public ObjectModel(String name, String address, int
age) {
super();
this.name = name;
this.address = address;
this.age = age;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getAddress() {
return address;
}
public void setAddress(String address) {
this.address = address;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public String toString() {
return name+" @
"+age+" @ "+address;
}
}
package memcached;
import java.io.IOException;
import java.util.Scanner;
import
java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;
/**
* The Class MemcachedTestClient.
*
* @author Abhinav
*/
public class MemcachedTestClient {
public static void main(String[] args) throws IOException,
InterruptedException,
ExecutionException {
UserModel
objMdl = new UserModel("abhinav", "noida", 25);
String commandName = "";
String keyName = "";
String value = "";
int expTimeInSeconds = 0;
long casIdentifier = 0;
Scanner
sc = new Scanner(System.in);
System.out.println("Enter command: ");
commandName
= sc.nextLine();
System.out.println("Enter keyName: ");
keyName
= sc.nextLine();
if (!(commandName.equals("get") || commandName.equals("delete") || commandName
.equalsIgnoreCase("getObject"))) {
if (!(commandName.equalsIgnoreCase("addObject"))) {
System.out.println("Enter value: ");
value
= sc.nextLine();
}
if (commandName.equals("append") || commandName.equals("prepend")) {
System.out.println("Enter casIdentifier:
");
casIdentifier
= sc.nextInt();
}
else {
System.out.println("Enter expiryTime(in
seconds): ");
expTimeInSeconds
= sc.nextInt();
}
}
MemcachedClient memcachedClient = new MemcachedClient(AddrUtil
.getAddresses("localhost:11211"));// getting connection from
//
server
long startTime = System.nanoTime();
if (commandName.equals("get")) {
/**
* It is used to get the value of the key from a Memcached server.
* If you have stored complex objects then the client will first
* retrieve the value and then deserialize it and return the object.
* The get() method returns null if the value is not found or is
* expired.
*/
System.out.println("Key Name: " + keyName);
System.out.println("Value of key: " + memcachedClient.get(keyName));
}
else if (commandName.equals("set")) {
/**
* MemcachedClient.set()
*
* You can use MemcachedClient.set() to store a simple string or a
* complex object. When you store a complex object, MemcachedClient
* will first serialize the object and then store it. As a result,
* every object that you store in Memcached must be serializable,
* and the key must also be a string. The set() method returns an
* object of Future. When we call the set method, it is executed
* asynchronously, so the control moves to the next line without
* waiting for a response from the Memcached server. If you needed
* to know the result of the set operation, then you would call
* setResult.get() on the Future object instance.
*
* @param: String
keyName
* @param: int expiration value (i.e, the expiration of cached
* object
in seconds, default is 0 seconds, it means cached
* object
will not expire until we delete it.)
* @param: value
* */
System.out.println("Key Name: " + keyName + " value: " + value
+
"
expTimeInSeconds of this object: " + expTimeInSeconds);
Future<Boolean>
result = memcachedClient.set(keyName,
expTimeInSeconds,
value);
System.out.println("Result of set operation:
" + result.get());
}
else if (commandName.equals("add")) {
/**
* It is used to add an object to the cache only if it does not
* already exist. Any complex object will be serialized before
* adding to cache. So every object that you store in Memcached must
* be serializable.
*
* @param: String
keyName
* @param: int expiration value (i.e, the expiration of cached
* object
in seconds, default is 0 seconds, it means cached
* object
will not expire until we delete it.)
* @param: value
*/
System.out.println("Key Name: " + keyName + " value: " + value
+
"
expTimeInSeconds of this object: " + expTimeInSeconds);
Future<Boolean>
result = memcachedClient.add(keyName,
expTimeInSeconds,
value);
System.out.println("Result of add operation:
" +
result.get());
}
else if (commandName.equals("replace")) {
/**
* It replaces an object with the value for the given key (if there
* is already such a value).
*
* @param: String
keyName
* @param: int expiration value (i.e, the expiration of cached
* object
in seconds, default is 0 seconds, it means cached
* object
will not expire until we delete it.)
* @param: value
*/
System.out.println("Key Name: " + keyName + " value: " + value
+
"
expTimeInSeconds of this object: " + expTimeInSeconds);
Future<Boolean>
result = memcachedClient.replace(keyName,
expTimeInSeconds,
value);
System.out.println("Result of replace
operation: " +
result.get());
}
else if (commandName.equals("delete")) {
/**
* It deletes a given key from the cache.
* */
System.out.println("Key Name: " + keyName);
Future<Boolean>
result = memcachedClient.delete(keyName);
System.out.println("Result of delete
operation: " +
result.get());
}
else if (commandName.equals("append")) {
/**
* Append (add value at last position of your previous value )to an
* existing value in the cache. prepend
*
* @param: long casIdentifier
* @param: String
keyName
* @param: value
* */
System.out.println("Key Name: " + keyName + " value: " + value
+
"
casIdentifier of this object: " + casIdentifier);
Future<Boolean>
result = memcachedClient.append(casIdentifier,
keyName,
value);
System.out.println("Result of append
operation: " +
result.get());
}
else if (commandName.equals("prepend")) {
/**
* Prepend(add value at start position of your previous value) from
* an existing value in the cache.
*
* @param: long casIdentifier
* @param: String
keyName
* @param: value
* */
System.out.println("Key Name: " + keyName + " value: " + value
+
"
casIdentifier of this object: " + casIdentifier);
Future<Boolean>
result = memcachedClient.prepend(casIdentifier,
keyName,
value);
System.out.println("Result of prepend
operation: " +
result.get());
}
else if (commandName.equals("addObject")) {
/**
* It is used to add an object to the cache only if it does not
* already exist. Any complex object will be serialized before
* adding to cache. So every object that you store in Memcached must
* be serializable.
*
* @param: String
keyName
* @param: int expiration value (i.e, the expiration of cached
* object
in seconds, default is 0 seconds, it means cached
* object
will not expire until we delete it.)
* @param: value
*/
System.out.println("Key Name: " + keyName
+
"
expTimeInSeconds of this object: " + expTimeInSeconds);
Future<Boolean>
result = memcachedClient.add(keyName,
expTimeInSeconds,
objMdl);
System.out
.println("Result of addObject
operation: " +
result.get());
}
else if (commandName.equalsIgnoreCase("getObject")) {
/**
* It is used to get the value of the key from a Memcached server.
* If you have stored complex objects then the client will first
* retrieve the value and then deserialize it and return the object.
* The get() method returns null if the value is not found or is
* expired.
*/
System.out.println("Key Name: " + keyName);
System.out
.println("Object of key: " + memcachedClient.get(keyName));
}
else {
System.out.println("Command not found.");
}
long endTime = System.nanoTime();
System.out.println("Total elapsed time in
nanoseconds: "
+
(endTime - startTime));
memcachedClient.shutdown();
}
}
Output>>
Add Object>>>>
Enter
command:
addObject
Enter
keyName:
abhinav
Enter
expiryTime(in seconds):
120
2012-05-29
15:52:12.820 INFO Key Name: abhinav expTimeInSeconds of this object: 120
net.spy.memcached.MemcachedConnection: Added {QA sa=localhost/127.0.0.1:11211,
#Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to
connect queue
2012-05-29
15:52:12.820 INFO net.spy.memcached.MemcachedConnection: Connection state changed for
sun.nio.ch.SelectionKeyImpl@9971ad
Result
of addObject operation: true
Total
elapsed time in nanoseconds: 21200869
2012-05-29
15:52:12.851 INFO net.spy.memcached.MemcachedClient: Shut down memcached client
Get Object>>>>
Enter
command:
getObject
Enter
keyName:
abhinav
2012-05-29
15:52:31.883 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=localhost/127.0.0.1:11211,
#Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to
connect queue
Key
Name: abhinav
2012-05-29
15:52:31.915 INFO net.spy.memcached.MemcachedConnection: Connection state changed for
sun.nio.ch.SelectionKeyImpl@9971ad
Object
of key: abhinav @ 25 @ noida
Total
elapsed time in nanoseconds: 24881729
2012-05-29
15:52:31.915 INFO net.spy.memcached.MemcachedClient: Shut down memcached client
Starting, Stopping, restarting and deleting server instance:
Ø
Right click on any instance, click Stop Service
[to stop], Restart Service [to restart] ,Start service [to start] and Delete
instance [to delete].
Add another instance,re-deployee/update memcached and Delete
Server:
Ø
Right click on server, click Add New Instance
Service [to add new instance server], Redeploy/Update MemCacheD [to redeploy or
update], Delete Server [to delete the server] and Refresh [to refresh the
server list].
We can also use Binary version of this server:
Ø
Download the binary version as indicated link
above.
Ø
Extract zip file.
Ø
Just run the “memcached.exe” file.
It looks like this:
Ø
We can now use it directly.
Ø
It is default configured on localhost at port no. “11211”
with default RAM memory as 64 megabyte.
References: http://en.wikipedia.org/wiki/Memcached
Leave your comments/suggestions below, Otherwise the next time :)
References: http://en.wikipedia.org/wiki/Memcached
Leave your comments/suggestions below, Otherwise the next time :)
