How to generate memcached_combined_ir.bc for memcached?

[egnchen]

Hello Technion researchers. I guess it has been a while since anyone post questions on this repo.

I want to make some changes to the memcached code and see how things behave, but currently memcached compilation is done through linking temp.c with pre-compiled(and pre-distributed) memcached_combined_ir.bc bytecode file. I wonder how I can regenerate the bytecode file by myself. Is there any guide on this?

[shmeni]

Thanks for your interest in cosmix! To try and answer the questions:

• Memcached was precompiled to a combined bytecode file indeed. Here's an online guide on how to do that (it has good integration with autotool): https://llvm.org/docs/GoldPlugin.html
• With the combined bytecode file you should be able to compile with cosmix's mstores, see the Cosmix.mk file in the Memcached folder for a reference on how we did that. Specifically, you should focus on the memcached_suvm compilation rule.
• Please also note that the temp.c file is only used for native compilation, which can be achieved with the traditional Memcached makefile and it is not required for working with cosmix's mstores.

Please let us know if you have any other questions

[egnchen]

Thanks for your timely reply. Following the instructions given I generated the combined IR file with

# *.bc gives 12 files in total
llvm-link *.bc -o memcached_combined_ir.bc

After that I re-compiled the memcached with suvm configuration and tried to run some test. memcached_suvm segfaulted after ~10 requests from memaslap.

GDB:

Thread 4 "memcached_suvm" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff702e9700 (LWP 16099)]
__memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:316
316	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
(gdb) bt
#0  __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:316
#1  0x00005555555891a3 in drive_machine (c=0x7fff58026460) at memcached.c:4210
#2  0x00007ffff79818f8 in ?? () from /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6
#3  0x00007ffff798233f in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6
#4  0x00005555555aab13 in worker_libevent (arg=0x555555873200) at thread.c:377
#5  0x00007ffff7bbb6db in start_thread (arg=0x7fff702e9700) at pthread_create.c:463
#6  0x00007ffff70f261f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

So something is wrong with memmove. Somehow CosMIX didn't replace the memmove implementation, and while the first argument is a cosmix pointer, it is passed directly into the memmove asm implementation. How do I fix this?

Also, some weird things I found during CosMIX pass in compilation:

Use original memcached_combined_ir.bc:

[INFO] replacing alloc func: __cosmix_malloc_suvm
[INFO] replacing alloc func: __cosmix_malloc_suvm
[DBG] Running pointer analysis in function: do_slabs_newslab for annotated var:   %74 = tail call noalias i8* @__cosmix_malloc_suvm(i64 %70) #11, !dbg !3477
[DBG] Running pointer analysis in function: slabs_prefill_global for annotated var:   %18 = tail call noalias i8* @__cosmix_malloc_suvm(i64 %7) #11, !dbg !3398
[INFO] Total number of allocation calls instrumented 2
[INFO] Total number of memory access instructions: 5097; Instrumented memory instructions are : 21%
[INFO] Total number of memory instructions accessed in loops: 381; #optimized: 35

Using newly-generated combined ir:

[INFO] replacing alloc func: __cosmix_malloc_suvm
[DBG] Running pointer analysis in function: do_slabs_newslab for annotated var:   %call1.i = tail call noalias i8* @__cosmix_malloc_suvm(i64 %conv16) #3, !dbg !1936
[INFO] Total number of allocation calls instrumented 1
[INFO] Total number of memory access instructions: 3639; Instrumented memory instructions are : 15%
[INFO] Total number of memory instructions accessed in loops: 199; #optimized: 25

So there should be a function called slabs_prefill_global in the source code that should be analyzed by the pass but it is not there. Apparently there is a version mismatch between the memcached source code provided and your original memcached_combined_ir.bc, so could you tell me which version of memcached are we supposed to be looking into?