redis analyst (7)- redis server/client source code analyst

redis/jedis虽然代码比较少,但是很难一篇文章概况所有设计及实现细节,所以还是抓取日常工作中可能的疑问,带着问题去翻代码:

1 假设当前内存占用是10G,那么落到磁盘rdb时,是否一定也是10G?

不一定,因为一方面可以设置压缩(默认开启:rdbcompression yes),另外一方面,假设内存中存在大量已过期数据,则也过滤掉这部分数据, dump的实现上,会遍历所有数据库(0-默认16),遍历所有数据然后保存:

int rdbSaveKeyValuePair(rio *rdb, robj *key, robj *val,
                        long long expiretime, long long now)
{
    /* Save the expire time */
    if (expiretime != -1) {
        /* If this key is already expired skip it */
        if (expiretime < now) return 0;
        if (rdbSaveType(rdb,RDB_OPCODE_EXPIRETIME_MS) == -1) return -1;
        if (rdbSaveMillisecondTime(rdb,expiretime) == -1) return -1;
    }

    /* Save type, key, value */
    if (rdbSaveObjectType(rdb,val) == -1) return -1;
    if (rdbSaveStringObject(rdb,key) == -1) return -1;
    if (rdbSaveObject(rdb,val) == -1) return -1;
    return 1;
}

2 monitor实现

void monitorCommand(client *c) {
    /* ignore MONITOR if already slave or in monitor mode */
    if (c->flags & CLIENT_SLAVE) return;

    c->flags |= (CLIENT_SLAVE|CLIENT_MONITOR);
    listAddNodeTail(server.monitors,c); //把整个客户端增加到server.monitors里面去
    addReply(c,shared.ok);
}

执行命令时:

void call(client *c, int flags) {
    long long dirty, start, duration;
    int client_old_flags = c->flags;

    /* Sent the command to clients in MONITOR mode, only if the commands are
     * not generated from reading an AOF. */
    if (listLength(server.monitors) &&
        !server.loading &&
        !(c->cmd->flags & (CMD_SKIP_MONITOR|CMD_ADMIN)))
    {
        replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc); //把命令发到client去
    }

可以根据info来查询输出list的最大值。或者根据client list也可以排查问题

client_longes_output_list信息获取的方法,遍历所有client,获许对应的值,取最大值。

void getClientsMaxBuffers(unsigned long *longest_output_list,
                          unsigned long *biggest_input_buffer) {
    client *c;
    listNode *ln;
    listIter li;
    unsigned long lol = 0, bib = 0;

    listRewind(server.clients,&li);
    while ((ln = listNext(&li)) != NULL) {
        c = listNodeValue(ln);

        if (listLength(c->reply) > lol) lol = listLength(c->reply);
        if (sdslen(c->querybuf) > bib) bib = sdslen(c->querybuf);
    }
    *longest_output_list = lol;
    *biggest_input_buffer = bib;
}

Eclipse luna with nuwen: Toolchain “MinGW GCC” is not detected

安装完最新版(luna)的eclipse cpp(或eclipse java with cdt plugin)和nuwen后,新建c++ project并没有显示支持mingw toolchain. 打开属性有所提示“Toolchain “MinGW GCC” is not detected”。

而之前的一些eclipse版本是支持的,且没有任何特殊配置。

通过eclipse源码分析可知:新的eclipse由简单的check mingw的路径存在与否改变成check对应32版本的mingw32-gcc.exe和对应64版本的x86_64-w64-mingw32-gcc.exe,而nuwen安装之后存在的仅仅是gcc.exe,将此文件复制成对应的文件名即可呈现支持mingw toolchain.所以这个问题是由eclipse本身的代码改变所引起,与其他无关。

同时从源码可以看出,eclipse 32仅匹配mingw 32,而eclipse 64可以匹配mingw 32和mingw 64.

MinGW 定位的全部源码

新版本的eclipse源码:


	@Override
	public boolean isSupported(IToolChain toolChain, Version version, String instance) {
		IEnvironmentVariable var = new EnvironmentVariableManagerToolChain(toolChain).getVariable(ENV_PATH, true);
		String envPath = var != null ? var.getValue() : null;
		return MinGW.isAvailable(envPath);
	}


}

	/**
	 * Check if MinGW is available in the path.
	 *
	 * @param envPath - list of directories to search for MinGW separated
	 *    by path separator (format of environment variable $PATH)
	 *    or {@code null} to use current $PATH.
	 * @return {@code true} if MinGW is available, {@code false} otherwise.
	 */
	public static boolean isAvailable(String envPath) {
		return isWindowsPlatform && findMingwInPath(envPath) != null;
	}

	private static String findMingwInPath(String envPath) {
		if (envPath == null) {
			// $PATH from user preferences
			IEnvironmentVariable varPath = CCorePlugin.getDefault().getBuildEnvironmentManager().getVariable(ENV_PATH, null, true);
			if (varPath != null) {
				envPath = varPath.getValue();
			}
		}

		String mingwLocation = mingwLocationCache.get(envPath);
		// check if WeakHashMap contains the key as null may be the cached value
		if (mingwLocation == null && !mingwLocationCache.containsKey(envPath)) {
			// Check for MinGW-w64 on Windows 64 bit, see http://mingw-w64.sourceforge.net/
			if (Platform.ARCH_X86_64.equals(Platform.getOSArch())) {
				IPath gcc64Loc = PathUtil.findProgramLocation("x86_64-w64-mingw32-gcc.exe", envPath); //$NON-NLS-1$
				if (gcc64Loc != null) {
					mingwLocation  = gcc64Loc.removeLastSegments(2).toOSString();
				}
			}

			// Look for mingw32-gcc.exe
			if (mingwLocation == null) {
				IPath gccLoc = PathUtil.findProgramLocation("mingw32-gcc.exe", envPath); //$NON-NLS-1$
				if (gccLoc != null) {
					mingwLocation = gccLoc.removeLastSegments(2).toOSString();
				}
			}
			mingwLocationCache.put(envPath, mingwLocation);
		}

		return mingwLocation;
	}

旧版本eclipse代码:


public class MingwIsToolChainSupported implements IManagedIsToolChainSupported {
	@Override
	public boolean isSupported(IToolChain toolChain, Version version, String instance) {
		// Only supported if we can find the mingw bin dir to run the compiler
		return MingwEnvironmentVariableSupplier.getBinDir() != null;
	}