修复集合方法extend使用错误的提示信息 - #7
Open
HuangSongZ wants to merge 1 commit into
Open
Conversation
Contributor
|
Great! |
yao-xiaobai
pushed a commit
that referenced
this pull request
Jul 26, 2025
修复 reindex 列存表出现 heap_use_after_free 问题 Created-by: sungwu_admin Commit-by: wusong Merged-by: opengauss_bot Description: 【标题】修复 reindex 列存表出现 heap_use_after_free 问题 【实现内容】: 修改流程，将是否是唯一索引的判断提前，避免出现 indexRelation 指针所指向内存已经被释放，仍然访问问题 【根因分析】: 由于流程中出现了释放 index lock 的操作，可能导致 index 被其他会话修改，从而导致本线程中系统缓存失效。再次使用时出现访问已被释放内存问题。 从 memcheck 信息看到，indexRel 被释放是在 reindex_index 中执行 index_open 对索引加锁时，收到 invalidation 消息，触发缓存失效导致。 #3 0x1fcb0f2 in pfree(void*) src/common/backend/utils/mmgr/mcxt.cpp:1263 #4 0x1c7c5e4 in RelationDestroyRelation(RelationData*, bool) src/common/backend/utils/cache/relcache.cpp:3635 #5 0x1c7d40d in RelationClearRelation(RelationData*, bool) src/common/backend/utils/cache/relcache.cpp:3745 #6 0x1c7fdea in RelationCacheInvalidate() src/common/backend/utils/cache/relcache.cpp:4064 #7 0x1c3444a in InvalidateSessionSystemCaches() src/common/backend/utils/cache/inval.cpp:783 #8 0x1c34422 in InvalidateSystemCaches() src/common/backend/utils/cache/inval.cpp:776 #9 0x4f4be75 in ReceiveSharedInvalidMessages(void (*)(SharedInvalidationMessage*), void (*)(), bool) src/gausskernel/storage/ipc/sinval.cpp:173 #10 0x1c34f24 in AcceptInvalidationMessages() src/common/backend/utils/cache/inval.cpp:906 #11 0x4f65894 in LockRelationOid(unsigned int, int) src/gausskernel/storage/lmgr/lmgr.cpp:89 #12 0x423a2c1 in relation_open(unsigned int, int, short) src/gausskernel/storage/access/heap/heapam.cpp:1490 #13 0x435ffcb in index_open(unsigned int, int, short) src/gausskernel/storage/access/index/indexam.cpp:168 【实现方案】: 将索引是否是唯一索引的访问提前，在持有锁时进行判断。 【关联需求或issue】: https://gitcode.com/opengauss/openGauss-server/issues/299 【开发自验报告】: 1. 请附上自验结果(内容或者截图) 2. 是否可以添加fastcheck测试用例，如是，请补充fastcheck用例 3. 是否涉及资料修改，如是，在docs仓库补充资料 4. 是否考虑升级场景(系统表修改、日志持久化以及修改执行态数据格式) 5. 是否考虑在线扩容等扩展场景 6. 是否考虑异常场景/并发场景/前向兼容/性能场景 7. 是否对其他模块产生影响 【其他说明】: See merge request: opengauss/openGauss-server!7775
yao-xiaobai
pushed a commit
that referenced
this pull request
Aug 11, 2025
将cfunchash从全局变量改为u_sess变量，去除大部分使用g_dlerror_lock_rw的地方 Created-by: totaj Commit-by: totaj Merged-by: opengauss_bot Description: 【标题】(请简要描述下实现的内容) 将cfunchash从全局变量改为u_sess变量，去除g_dlerror_lock_rw锁 【实现内容】: 将cfunchash从全局变量改为u_sess变量，去除g_dlerror_lock_rw锁 【根因分析】: 书接上回，在 https://gitcode.com/opengauss/openGauss-server/pull/8160 中，将 dlerror_lock 从mutex lock 改为了 rwlock，提升并发性，当时为了尽量少改动原始逻辑，虽然发现了 dlerror_lock 和 file_list 存在功能重复的问题，但是并没有在PR中进行修改。原始代码中， dlerror_lock 用来控制 file_list 和 CFuncHash 两个全局变量的并发读写，而 file_list_lock，单独控制 file_list，原因应该是部分load library的接口不在dlerror_lock的控制范围内，所以又新增了一个锁。现优化这两个锁的资源控制情况。 【实现方案】: * dlerror_lock用于控制cfunchash全局变量，file_list_lock用于控制 file_list 全局变量，两者范围不重复 * cfunchash从全局变量改为 u_sess 变量，去除大部分使用dlerror_lock锁，读写cfunchash不再需要锁保护，进一步提升并发性。当前唯一保留用 dlerror_lock 的地方是 ProcedureCreate 代码里面，为了保护 copyLibraryToSpecialName 并发重命名lib过程。由于只有这一个地方用dlerror_lock，且需要加写锁，所以把 dlerror_lock 改回mutex锁。因为只有写冲突场景，用mutex效率更高。 * internal_load_library 函数中，原来通过先加读锁，后加写锁，再改为读锁的方式来处理第一次加载so的场景，在第一次并发的场景，都释放读锁改为加写锁后，若成功加载了so，原先代码又将释放写锁，改回读锁。但这里可能有问题，若有其他会话在当前会话释放写锁到获取读锁之间拿到了写锁，并修改了 file_list，此时记录的file_scanner可能已被其他人修改，固这里不能释放写锁，需一直持有写锁，直到file_scanner的访问完成。所以删除了这部分逻辑。 * cfunchash的内存从g_instance.instance_context改为u_sess->cache_mem_cxt，和会话的生命周期保持一致。 影响分析： 将cfunchash从全局变量改为u_sess变量，完全去除锁访问，对性能有提升优化。但是对内存使用会造成影响，原先不同会话加载同一个so的同一个函数时，会统一存在cfunchash全局变量中，只存储一份，但是他的生命周期是随进程一直存在。改成u_sess后，每个会话有自己的hash表，存在重复记录的问题，但是内存会在会话结束后释放，不会一直驻留。潜在的可能增强的内存消耗，按 1000 并发计算，当前dolpin中一共有 1600+ C函数，假设所有函数均被调用需要缓存，当前一个hash entry的大小为 40 字节，在原代码场景，所有函数均存在一个全局hash中，和并发量无关，占用内存约为： 40 * 1600 = 64000 = 62.5 K（不考虑hashtable额外的内存，只考虑entry自己的内存）。改为一个会话一个hash之后，占用内存为 62.5 K * 1000（并发） = 62500K = 61M，会多占用 61M 内存（实际可能更多点，hashtab自带的消耗），这个场景比较极端，如果最大并发减到500，考虑1600个函数中，常用的可能只占50%甚至更少的话，占用的内存在 61M / 2 / 2 = 15M 左右。 【关联需求或issue】: https://gitcode.com/opengauss/Plugin/issues/1521 【开发自验报告】: 1. 请附上自验结果(内容或者截图) 优化前，B库B协议 sysbench 128并发，read only， 35 W QPS  优化后，B库B协议 sysbench 128并发，read only， 35.4 W QPS  2. 是否可以添加fastcheck测试用例，如是，请补充fastcheck用例 否。性能优化，对外表现无变更 3. 是否涉及资料修改，如是，在docs仓库补充资料 不涉及 4. 是否考虑升级场景(系统表修改、日志持久化以及修改执行态数据格式) 不涉及 5. 是否考虑在线扩容等扩展场景 不涉及 6. 是否考虑异常场景/并发场景/前向兼容/性能场景 不涉及 7. 是否对其他模块产生影响 不涉及 8. 是否需要回合补丁版本(5.0、6.0)，涉及(**结果错误|coredump|性能劣化|内存泄露|集群异常**)相关修复请排查回合补丁分支 不涉及 【其他说明】: assign_shared_preload_libraries函数的修改跟本PR的原始目的无关，是跑门禁的时候经常出现core问题，本地多次连跑发现了如下core： ``` (gdb) bt #0 __strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:181 #1 0x00000000013ab5f6 in MemoryContextStrdupDebug (context=0x7f552ca4b9d8, string=0x7f5633cbc488 'Z' <repeats 200 times>..., file=0x79a720c "guc.cpp", line=10935) at mcxt.cpp:1881 #2 0x00000000013703b9 in _ShowOption (record=0x7f5512400920, use_units=false, is_show=true) at guc.cpp:10935 #3 0x000000000136da31 in GetConfigOptionByNum (varnum=707, values=0x7f55a34b5470, noshow=0x7f55a34b541f) at guc.cpp:10299 #4 0x000000000136f525 in show_all_settings (fcinfo=0x7f55a34b56e0) at guc.cpp:10699 #5 0x0000000001cbca9d in ExecMakeTableFunctionResult (funcexpr=0x7f5598efa220, econtext=0x7f552a5e0288, expectedDesc=0x7f5598ef9088, randomAccess=false, node=0x7f552a5e0060) at execQual.cpp:3315 #6 0x0000000001d6b68c in FunctionNext (node=0x7f552a5e0060) at nodeFunctionscan.cpp:63 #7 0x0000000001ce159d in ExecScanFetch (node=0x7f552a5e0060, access_mtd=0x1d6b5fa <FunctionNext(FunctionScanState*)>, recheck_mtd=0x1d6b6d6 <FunctionRecheck(FunctionScanState*, TupleTableSlot*)>) at execScan.cpp:83 #8 0x0000000001ce172c in ExecScan (node=0x7f552a5e0060, access_mtd=0x1d6b5fa <FunctionNext(FunctionScanState*)>, recheck_mtd=0x1d6b6d6 <FunctionRecheck(FunctionScanState*, TupleTableSlot*)>) at execScan.cpp:172 #9 0x0000000001d6b713 in ExecFunctionScan (state=0x7f552a5e0060) at nodeFunctionscan.cpp:96 #10 0x0000000001cb4c8b in ExecProcNodeFirst (node=0x7f552a5e0060) at execProcnode.cpp:800 #11 0x0000000001ca7df3 in ExecProcNode (node=0x7f552a5e0060) at ../../../../src/include/executor/executor.h:297 #12 0x0000000001cace78 in ExecutePlan (estate=0x7f558ede4060, planstate=0x7f552a5e0060, operation=CMD_SELECT, sendTuples=true, numberTuples=0, direction=ForwardScanDirection, dest=0x7f5577bf6728, motJitContext=0x0) at execMain.cpp:2300 #13 0x0000000001ca9362 in standard_ExecutorRun (queryDesc=0x7f5598e12c60, direction=ForwardScanDirection, count=0) at execMain.cpp:684 #14 0x0000000001a509a3 in explain_ExecutorRun (queryDesc=0x7f5598e12c60, direction=ForwardScanDirection, count=0) at auto_explain.cpp:122 #15 0x00000000015110e2 in instr_trace_ExecutorRun (queryDesc=0x7f5598e12c60, direction=ForwardScanDirection, count=0) at instr_trace.cpp:708 #16 0x0000000001ca8d0b in ExecutorRun (queryDesc=0x7f5598e12c60, direction=ForwardScanDirection, count=0) at execMain.cpp:523 #17 0x0000000001be9c25 in PortalRunSelect (portal=0x7f55859a0060, forward=true, count=0, dest=0x7f5577bf6728) at pquery.cpp:1493 #18 0x0000000001be8c4e in PortalRun (portal=0x7f55859a0060, count=9223372036854775807, isTopLevel=true, dest=0x7f5577bf6728, altdest=0x7f5577bf6728, completionTag=0x7f55a34b6310 "") at pquery.cpp:1211 #19 0x0000000001bcad6f in exec_simple_query (query_string=0x7f5574b44060 "select setting from pg_settings where name = 'dolphin.lower_case_table_names';", messageType=QUERY_MESSAGE, msg=0x7f55a34b66f0) at postgres.cpp:3186 #20 0x0000000001bd9ada in PostgresMain (argc=1, argv=0x7f559f9078b8, dbname=0x0, username=0x0) at postgres.cpp:9756 #21 0x0000000001b05d87 in BackendRun (port=0x7f55a34b7c00) at postmaster.cpp:9987 #22 0x0000000001b18b31 in GaussDbThreadMain<(knl_thread_role)2> (arg=0x7f56323c6360) at postmaster.cpp:14710 #23 0x0000000001b137ee in InternalThreadFunc (args=0x7f56323c6360) at postmaster.cpp:15384 #24 0x00000000028d372d in ThreadStarterFunc (arg=0x7f56323c6350) at gs_thread.cpp:383 #25 0x00007f5643176f2b in start_thread (arg=0x7f55a34ef700) at pthread_create.c:486 #26 0x00007f56430ae33f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb) f 2 #2 0x00000000013703b9 in _ShowOption (record=0x7f5512400920, use_units=false, is_show=true) at guc.cpp:10935 10935 return pstrdup(val); (gdb) p *record $1 = {name = 0x79a8dfd "shared_preload_libraries", context = PGC_POSTMASTER, nodetype = NODE_ALL, group = RESOURCES_KERNEL, short_desc = 0x79a8e18 "Lists shared libraries to preload into server.", long_desc = 0x0, flags = 257, vartype = PGC_STRING, status = 0, source = PGC_S_DEFAULT, reset_source = PGC_S_DEFAULT, scontext = PGC_INTERNAL, reset_scontext = PGC_INTERNAL, stack = 0x0, extra = 0x0, sourcefile = 0x0, sourceline = 0} (gdb) p g_instance.attr.attr_common.shared_preload_libraries_string $2 = 0x7f5633cbc488 'Z' <repeats 200 times>... ``` 应该是这个shared_preload_libraries guc参数的内存出问题了，参考其他修改，改为 guc_xxx 相关的内存分配函数，使用cbb内存上下文，而不是currentmemorycontext See merge request: opengauss/openGauss-server!8185
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DECLARE -- Associative array indexed by string: TYPE population IS TABLE OF NUMERIC -- Associative array type INDEX BY VARCHAR(64); -- indexed by string city_population population; begin city_population('Smallville') := 2000; city_population.extend; -- EXTEND is not support the Associative array end; /ERROR: index by varchar type don't support trim function at or near "city_population"
这个提示信息是有问题的，用例使用的是extend方法，但是执行提示是使用了trim方法。