I created a new environmental variable using the LD_LIBRARY_PATH variable:
$ export LD_LIBRARY_PATH=/opt/R/3.4.1/lib64/R/lib
When I re-run ldd R_X11.so, it seems to find the newly compiled libR.so file:
$ ldd R_X11.so
linux-vdso.so.1 => (0x00007ffeda1e2000)
libjpeg.so.62 => /lib64/libjpeg.so.62 (0x00007f12c1f58000)
libpng15.so.15 => /lib64/libpng15.so.15 (0x00007f12c1d2c000)
libSM.so.6 => /lib64/libSM.so.6 (0x00007f12c1b24000)
libICE.so.6 => /lib64/libICE.so.6 (0x00007f12c1908000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007f12c15c9000)
libXt.so.6 => /lib64/libXt.so.6 (0x00007f12c1362000)
libR.so => /opt/R/3.4.1/lib64/R/lib/libR.so (0x00007f12c0d33000)
libm.so.6 => /lib64/libm.so.6 (0x00007f12c0a30000)
libc.so.6 => /lib64/libc.so.6 (0x00007f12c066d000)
libz.so.1 => /lib64/libz.so.1 (0x00007f12c0457000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f12c0251000)
libxcb.so.1 => /lib64/libxcb.so.1 (0x00007f12c0029000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f12bfe25000)
libRblas.so => /opt/R/3.4.1/lib64/R/lib/libRblas.so (0x00007f12bfbf7000)
libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007f12bf8d5000)
libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f12bf699000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f12bf452000)
libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f12bf1f0000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f12befca000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f12bedb9000)
librt.so.1 => /lib64/librt.so.1 (0x00007f12bebb1000)
libicuuc.so.50 => /lib64/libicuuc.so.50 (0x00007f12be838000)
libicui18n.so.50 => /lib64/libicui18n.so.50 (0x00007f12be439000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f12be213000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f12bdff7000)
/lib64/ld-linux-x86-64.so.2 (0x0000555bcf2a5000)
libXau.so.6 => /lib64/libXau.so.6 (0x00007f12bddf2000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f12bdbdc000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f12bd9b1000)
libicudata.so.50 => /lib64/libicudata.so.50 (0x00007f12bc3dd000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f12bc0d5000)
Now I can run R:
$ /opt/R/3.4.1/lib64/R/bin/R
When I run capabilities(), it still fails to detect png, jpeg, cairo, etc.
> capabilities()
jpeg png tiff tcltk X11 aqua
FALSE FALSE FALSE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE TRUE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE TRUE FALSE TRUE TRUE TRUE