diff --git a/modules/savanna/modules/graph-development/images/disable-profile.png b/modules/savanna/modules/graph-development/images/disable-profile.png new file mode 100644 index 00000000..2eaa71b0 Binary files /dev/null and b/modules/savanna/modules/graph-development/images/disable-profile.png differ diff --git a/modules/savanna/modules/graph-development/images/enable-profile.png b/modules/savanna/modules/graph-development/images/enable-profile.png new file mode 100644 index 00000000..c7a7f5f1 Binary files /dev/null and b/modules/savanna/modules/graph-development/images/enable-profile.png differ diff --git a/modules/savanna/modules/graph-development/images/profile.png b/modules/savanna/modules/graph-development/images/profile.png new file mode 100644 index 00000000..11eb5497 Binary files /dev/null and b/modules/savanna/modules/graph-development/images/profile.png differ diff --git a/modules/savanna/modules/graph-development/nav.adoc b/modules/savanna/modules/graph-development/nav.adoc old mode 100644 new mode 100755 index 75226e59..238df1e4 --- a/modules/savanna/modules/graph-development/nav.adoc +++ b/modules/savanna/modules/graph-development/nav.adoc @@ -9,8 +9,9 @@ *** xref:load-data/load-from-other-sources.adoc[] **** xref:load-data/jdbc.adoc[] ** xref:design-schema/index.adoc[Design Schema] -** xref:gsql-editor/index.adoc[GSQL Editor] +** xref:gsql-editor/index.adoc[Query Editor] *** xref:gsql-editor/how2-edit-gsql-query.adoc[Edit Query] +*** xref:gsql-editor/how2-get-query-profile.adoc[Query Profile] ** xref:explore-graph/index.adoc[Explore Graph] *** xref:explore-graph/how2-use-pattern-search.adoc[Pattern Search] ** xref:advanced-features/index.adoc[Advanced Features] diff --git a/modules/savanna/modules/graph-development/pages/gsql-editor/how2-get-query-profile.adoc b/modules/savanna/modules/graph-development/pages/gsql-editor/how2-get-query-profile.adoc new file mode 100755 index 00000000..f156ae39 --- /dev/null +++ b/modules/savanna/modules/graph-development/pages/gsql-editor/how2-get-query-profile.adoc @@ -0,0 +1,206 @@ += How to get a GSQL Query Profile +:experimental: + +Query profiling is a new feature added in TigerGraph version 4.2.0. It aims to provide profiling data when query is either running or completed. + +User can activate query profiling to view and analyze the details of a query execution. Query profiling data can help troubleshoot performance bottlenecks throughout the execution of a query. + +You can analyze the generated actions derived from the original statements to identify potential optimizations through query rewriting. + +To diagnose bottlenecks, you can examine each generated action along with its associated metrics, such as execution time, the number of vertices and edges accessed, and memory usage. + +[TIP] +==== +This feature applies to GSQL Syntax, not OpenCypher Syntax for now. +==== + +== Query Mode + +.This feature applies to the following queries: + +. Installed DISTRIBUTED user-defined queries + +. Installed NON-distributed user-defined queries if installed with flag -SINGLE, eg: INSTALL QUERY -SINGLE q1. + +. 3 built-in endpoints: + + /kstep_expansion + + /allpaths + + /searchvertex + +For those queries, the profiling data contains detailed information including memory and topology, see section ”Profiling Data Explanation” below. + +For other queries, the profiling data will only contain some brief information, in field profile.overall. + + +== How to activate Profiling + +.We use profile flags to indicate query profiling mode. + +In 4.2.0, we only provide 1 flag: BASIC. +RESTPP Request + +We just need to add header -H "PROFILE: BASIC" to the request. For example: + + +---- +curl -H "PROFILE: BASIC" -X POST 'http://localhost:14240/restpp/query/testGraph/qtest' +---- +This header may be used with async mode, so we can fetch the real-time profiling data with /query_status: + + +---- +// Send async request with profiling activated +curl -H "GSQL-ASYNC: true" -H "PROFILE: BASIC" -X GET 'http://localhost:14240/restpp/query/testGraph/qtest' + + +// Get real-time profiling data +curl -X GET "http://localhost:14240/restpp/query_status?requestid=$reqid" +---- + +*GSQL* + + +In GSQL Shell, we can activate profiling for run query in 2 ways. + +RUN QUERY Option + +---- +GSQL > RUN QUERY -PROFILE BASIC qtest() +---- + +GSQL Session Parameter + + +---- +GSQL > SET PROFILE = "BASIC" +GSQL > RUN QUERY qtest() +---- + +*Query Editor* + +You can get query profile in Query Editor. + +. You can create and install the query in the Query Editor, select the query, enable profiling by clicking the button "Enable Profiling" in the top right corner of the Query Editor *(It's enabled by default)*. ++ +image::enable-profile.png[] + + +. After run the query, the profiling data will be generated automatically. ++ +image::profile.png[] + + + + +*Profiling Data Explanation* + +This section is to explain the profiling data. + +If profiling is activated for a query, there will be a field "profile" for profiling data, in the output json of query result, and /query_status output for real-time profiling. + +*Profile Query Stages Breakdown* + +We can use the following simple query to understand the Profile Query Stages. + + +---- +CREATE DISTRIBUTED QUERY qTest(INT temp) { + INT threshold = 90; // statementKey 1 (1 action) + IF temp > threshold THEN // statementKey 2 (2 child statements) + STRING new_state = "sweat"; // statementKey 2.1 (1 action) + vSet = SELECT s from person:s // statementKey 2.2 (3 actions: create_vset, vertex, reduce) + POST-ACCUM s.state = new_state; + END; +} +---- + +For run of the query qTest, we can split to the following stages: + +. Scheduling: the stage between TigerGraph server receives the request, and a thread is assigned to work on compute of the request. + +. Execution: it contains 3 sub stages below. + + .. Initialization: initialize global variables that will be used. For distributed query also include starting workers on all nodes. + + .. Processing: main control flow logic and run actions, including preparing topology, computing, shuffling. + + .. Finalization: construct and send query result. + + +There are 2 types of fine-grained stages within the Processing stage: + +. Statement: it represents one complete statement in original query. It can be graph-traversal, control-flow, or other types. A control-flow statement may contain child statements. + +. Action: it is a minimum execution unit in execution plan, from final transformed and optimized query. For example: vertex map action. A statement without any child statements must have child actions, and it can have 1 or more child actions. A statement with child statements does NOT have child actions. + + +== General Metrics +This section introduces general metrics in query profiling that exists in different levels (Query, Statement and Action). + +*ExecutionCount* + +The metric executionCount represents how many times a stage has been executed and completed. It does not count the on-going stage. + +*Time* + +This metric contains the following data: + +. startTime: start time of the stage. This field is omitted when executionCount > 1. + +. endTime: end time of the stage. This field is omitted when executionCount > 1. + +*Memory* + +Data for this metric is separated to different GPE nodes, with name in format "GPE__". + +For each GPE node, this metric only contains vertexAccumulator for now, which represents memory usage of VERTEX-attached accumulators, without EDGE-attached accumulators or Global accumulators. + +Notice the vertexAccumulator include internal variables not explicitly declared in original query, so it can be non-zero even there’s no VERTEX-attached accumulators declared. + +. vertexAccumulator + + .. totalPeakMB: peak memory usages throughout execution of this stage. + + .. totalFinalMB: memory usage at end time of last execution of this stage. + +*Topology* + +Data for this metric is separated to different GPE nodes, with name in format "GPE__". + +For each GPE node, this metric contains non-zeros values for following fields: + +. vertex + + .. readCount: count of accesses to vertices in this stage. There might be duplicate accesses to the same entity. + +. edge + + .. readCount: count of accesses to edges in this stage. There might be duplicate accesses to the same entity. + +. table + + .. rowCount: count of accesses to table rows in this stage. There might be duplicate accesses to the same entity. + +. join + + .. totalCount: count of total join operations in this stage. + + + +*Disable Query Profiling Feature* + +1. If the Query Profiling feature is causing issues, it can be disabled for all installed queries by adding DISABLE_PROFILE=true; to GSQL.BasicConfig.Env, then re-install all queries. +2. Operate in frontend Query Editor ++ +image::disable-profile.png[] + + +== Next Steps + +Next, learn to xref:savanna:graph-development:explore-graph/index.adoc[]. + +Or return to the xref:savanna:overview:index.adoc[Overview] page for a different topic. + diff --git a/modules/savanna/modules/workgroup-workspace/nav.adoc b/modules/savanna/modules/workgroup-workspace/nav.adoc old mode 100644 new mode 100755