[sheepdog] VDI r/w performance comparison test

[Liu Yuan]

On Wed, May 27, 2015 at 04:24:29PM +0800, yuyang wrote:
> Hello everyone,
> 
> Recently we need copy one VDI from sheepdog to another storage 
> system(such as sheepdog or RAID). There are 2 ways to choose,
> 1. dog vdi r | dog vdi w,(only for sheepdog, or temp file is needed)
> 2. lib client, which is on the way.
> 
> And then I did a R/W IO performance comparison.
> Now I want to share the results.
> 
> In this test, we read from sheepdog and write to the same sheepdog. Or we just call 
> it "copy task". For the sake of simplicity, the sheepdog only has one node.
> 
> The first step is to prepare the source VDI. The size of source VDI is 50GB,
> and the data is 10GB/25GB/40GB(we do three tests).
> 
> The task by lib client is done by the following code:
> 
> /****** copy.c *********/
> #include <stdio.h>
> #include <stdlib.h>
> #include <sheepdog sheepdog.h="">
> 
> #define BUFSIZE (1024*1024*128)  /**128M**/
> 
> void copy(struct sd_cluster *csrc, char *srcname,
> 		  struct sd_cluster *cdst, char *dstname)
> {
> 	int ret = 0;
> 	struct sd_vdi * srcv = sd_vdi_open(csrc, srcname, 0, NULL);
> 	if (!srcv) {
> 		fprintf(stderr, "open %s failed\n", srcname);
> 		return ;
> 	}
> 	uint64_t size = srcv->inode->vdi_size;
> 	ret = sd_vdi_create(cdst, dstname, size, 0);
> 	if (ret != SD_RES_SUCCESS) {
> 		fprintf(stderr, "create failed %s\n", sd_error2str(ret));
> 		return;
> 	}
> 	struct sd_vdi * dstv = sd_vdi_open(cdst, dstname, 0, NULL);
> 	if (!dstv) {
> 		fprintf(stderr, "open %s failed\n", dstname);
> 		return ;
> 	}
> 	char *buf = malloc(BUFSIZE);
> 	if ( !buf ) {
> 		printf("OOM\n");
> 		return ;
> 	}
> 	uint64_t offset = 0, left = size, onew;
> 	while(offset < size) {
> 		left = size - offset;
> 		onew = left > BUFSIZE ? BUFSIZE : left;
> 		memset(buf, 0, BUFSIZE);
> 	
> 		ret = sd_vdi_read(srcv, buf, onew, offset);
> 		if (ret != SD_RES_SUCCESS) {
> 			fprintf(stderr, " read error: offset=%"PRIx64, offset);
> 			return;
> 		}
> 		ret = sd_vdi_write(dstv, buf, onew, offset);
> 		if (ret != SD_RES_SUCCESS) {
> 			fprintf(stderr, " write error: offset=%"PRIx64, offset);
> 			return;
> 		}
> 		offset += onew;
> 	}
> 	free(buf);
> }
> 
> int main(int argc, char *argv[])
> {
> 		char host1[256] = {};
> 		char host2[256] = {};
> 		memset(host1, 0, 256);
> 		memset(host2, 0, 256);
> 		if (argc != 5) {
> 			printf("too small argument\n");
> 			printf("usage: copy srcIP srcvdiname dstIP dstvdiname\n");
> 			return 0;
> 		}
> 
> 		strcpy(host1, argv[1]);
> 		strcpy(host2, argv[3]);
> 
> 		strcat(host1, ":7000");
> 		strcat(host2, ":7000");
> 
> 		struct sd_cluster *c1 = sd_connect(host1);
> 		struct sd_cluster *c2 = sd_connect(host2);
> 
> 		if (!c1 || !c2) {
> 			printf("connect failed\n");
> 			return 0;
> 		}
> 		copy(c1, argv[2], c1, argv[4]);
> 		return 0;
> }
> 
> compile: gcc copy.c -lpthread -lsheepdog
> run it:
> ./a.out 127.0.0.1 $vlibsrc 127.0.0.1 $vlibdst
> 
> we record the runtime of the programme.
> 
> And the copy task by dog cmd is done by the following script:
> dog vdi create $vdogdst 50g
> dog vdi read $vdogsrc  | dog vdi write $vdogdst 0 50g
> 
> we also record the runtime.
> 
> Then we prepare 2 new source VDI in which there is 25GB(and then 40GB) 
> and repeat the steps above.
> The result is as follows:
> 
> 	data(GB)	lib/dog		time(s) 
> 	10			lib			101
> 	10			dog			239
> 
> 	25			lib			128
> 	25			dog			251
> 
> 	40			lib			134
> 	40			dog			249
> 
> As is shown in the table above, we can see that most of the time,
> the lib client can speed up to more than 50% in VDI r/w.

Thanks for the share. I guess he performance gain is linear to the buffer size.
Could you please add a dimension 'buffer size' such as following?

       128M    256M     512M    1024M
lib     x       y         z      ..
dog     x'      y'        z'     ..

Thanks,
Yuan