Skip to content

Commit 3fcbb47

Browse files
heywalterheywalter
committed
Fix Broken Links
1 parent f7eae7b commit 3fcbb47

1 file changed

Lines changed: 164 additions & 0 deletions

File tree

‎docs/cloud/prerequisites/cloud-databases/mongodb-atlas.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
keywords:
3+
- MongoDB Atlas
4+
- MongoDB Atlas real-time sync
5+
- MongoDB Atlas data integration
6+
- MongoDB Atlas incremental replication
7+
- MongoDB cloud migration
8+
- TapData MongoDB Atlas connector
9+
---
10+
11+
# MongoDB Atlas
12+
13+
14+
15+
[TapData](https://tapdata.io/) supports [MongoDB Atlas](https://www.mongodb.com/atlas) as a data source, enabling real-time data sync, incremental replication, and seamless cloud-to-local data integration.
16+
17+
MongoDB Atlas is a multi-cloud managed database platform. This document describes how to connect MongoDB Atlas in TapData and use it for data migration, analytics, and syncing to local warehouses.
18+
19+
<Head>
20+
<link rel="canonical" href="https://docs.tapdata.io/prerequisites/cloud-databases/mongodb-atlas" />
21+
</Head>
22+
23+
## Supported Versions
24+
25+
MongoDB Atlas 5.0.15
26+
27+
:::tip
28+
29+
When synchronrizing data between MongoDB, it is recommended source/target database are 5.0 and above for ensure data compatibility.
30+
31+
:::
32+
33+
## Preparations
34+
35+
Before establishing the connection, it is essential to complete the necessary preparations outlined in the provided article. These preparations may include authorizing an account and performing other relevant steps to ensure a smooth and secure connection.
36+
37+
1. Log in to [MongoDB Atlas](https://cloud.mongodb.com/v2).
38+
39+
2. Set up network access control to ensure network connectivity.
40+
41+
1. In the left navigation panel, click **Network Access**.
42+
43+
2. On the right, click **ADD IP ADDRESS**.
44+
45+
3. In the pop-up dialog, fill in the public address of the TapData Agent (CIDR format) and click **Confirm**.
46+
47+
![Set Network Whitelist](../../../images/atlas_add_ip_address.png)
48+
49+
3. Create an account and grant permissions for database connectivity.
50+
51+
1. In the left navigation panel, click **Database Access**.
52+
53+
2. On the right side of the page, click **ADD NEW DATABASE USER**.
54+
55+
3. In the pop-up dialog, select the authentication method and grant permissions.
56+
57+
![Create an account and authorize](../../../images/atlas_create_user.png)
58+
59+
In this case, we will use password authentication as an example to demonstrate the operation process. The permission selection instructions are as follows.
60+
61+
* **As a Source Database**: Select **Built-in Role** as **Only read any database**.
62+
63+
* **As a Target Database**: Select **Built-in Role** for **Read and Write to any Database**.
64+
65+
4. Click **Add User**.
66+
67+
4. Gets database connection information.
68+
69+
1. In the left navigation panel, click **Database**.
70+
71+
2. Locate the target database and click **Connect**.
72+
73+
3. In the pop-up dialog, select **Connect to your application** to get connection information, which will be used when connecting to the database.
74+
75+
![Get Connection Information](../../../images/atlas_obtain_connection.png)
76+
77+
## Connect to MongoDB Atlas
78+
79+
1. Log in to TapData platform.
80+
81+
2. In the left navigation panel, click **Connections**.
82+
83+
3. On the right side of the page, click **Create**.
84+
85+
4. In the pop-up **dialog**, select **MongoDB Atlas**.
86+
87+
5. Fill in the connection information for MongoDB Atlas on the redirected page, following the instructions provided below.
88+
89+
![MongoDB Connection Example](../../../images/mongodb_atlas_connection_setting.png)
90+
91+
* Connection Information Settings
92+
93+
* **Connection name**: Fill in a unique name that has business significance.
94+
95+
* **Connection type**: Supports MongoDB Atlas as a source or target database.
96+
97+
* **Connection method**: Fixed in **URI Mode**.
98+
99+
* **Database URI**: Fill in the Database URI connection information, and URI should include the username and password, which are concatenated in the format.
100+
101+
For example, the connection string may look like: ` mongodb+srv://tapdata:Tap123456@cluster****.mongodb.net/demodata?retryWrites=true&w=majority`
102+
:::tip
103+
Be sure to set the database you want to connect in the connection string, for example, in the above example set to **demodata**, otherwise, it will cause the connection to fail and prompt for an error: 'datbaseName can not be null'.
104+
:::
105+
106+
* **Connect using TLS/SSL**: Choose how you want to connect:
107+
108+
* **TSL/SSL connection:** In cases where your database is located in an inaccessible subnet, TapData Cloud offers the option to establish a connection through a separate server within the network. This server acts as a TSL/SSL channel to facilitate the connection to the database. This method enables connectivity to the database even when it is in a subnet that would otherwise be inaccessible.
109+
* **Direct connection**: TapData Cloud will connect directly to the database and you need to set up security rules to allow access.
110+
111+
* Advanced settings
112+
* **Contain table**: The default option is **All**, which includes all tables. Alternatively, you can select **Custom** and manually specify the desired tables by separating their names with commas (,).
113+
* **Exclude tables**: Once the switch is enabled, you have the option to specify tables to be excluded. You can do this by listing the table names separated by commas (,) in case there are multiple tables to be excluded.
114+
* **Agent settings**: Defaults to **Platform automatic allocation**, you can also manually specify an agent.
115+
* **Model load time**: If there are less than 10,000 models in the data source, their information will be updated every hour. But if the number of models exceeds 10,000, the refresh will take place daily at the time you have specified.
116+
* **Enable heartbeat table**: This switch is supported when the connection type is set as the **Source&Target** or **Source**. TapData Cloud will generate a table named **tapdata_heartbeat_table** in the source database, which is used to monitor the source database connection and task health.
117+
:::tip
118+
After referencing and starting the data replication/development task, the heartbeat task will be activated. At this point, you can click **View heartbeat task** to monitor the task.
119+
:::
120+
121+
6. Click **Test**, and when passed, click **Save**.
122+
123+
:::tip
124+
125+
If the connection test fails, follow the prompts on the page to fix it.
126+
127+
:::
128+
129+
130+
131+
## Node Advanced Features
132+
133+
```mdx-code-block
134+
import Tabs from '@theme/Tabs';
135+
import TabItem from '@theme/TabItem';
136+
```
137+
138+
When configuring data synchronization or transformation tasks with MongoDB Atlas as the source or target node, TapData provides more advanced features to better meet complex business needs and maximize performance. You can configure these features based on your requirements:
139+
140+
![MongoDB Node Advanced Features](../../../images/mongodb_atlas_node_advanced_settings.png)
141+
142+
```mdx-code-block
143+
<Tabs className="unique-tabs">
144+
<TabItem value="MongoDB Atlas as a Source Node">
145+
```
146+
147+
| **Configuration** | **Description** |
148+
| ------------------------- | ------------------------------------------------------------ |
149+
| **No Cursor Timeout** | Disabled by default. When enabled, MongoDB Atlas will disable cursor timeout to prevent timeout errors during data synchronization. |
150+
| **Filling Modified Data** | Enabled by default. When enabled, it automatically fills in complete fields for **UPDATE** events. |
151+
152+
153+
</TabItem>
154+
155+
<TabItem value="MongoDB Atlas as a Target Node">
156+
157+
| **Configuration** | **Description** |
158+
| ----------------------------- | ------------------------------------------------------------ |
159+
| **Sync Index** | Disabled by default. When enabled, indexes from the source database will be automatically synchronized to the target database during the full data synchronization phase. |
160+
| **Sync Partition Properties** | Disabled by default. When enabled, sharding attributes are kept consistent during synchronization between MongoDB Atlas sharded clusters. |
161+
| **Save Deleted Data** | Disabled by default. When enabled, deleted data content will be cached.<br/> <br />Since MongoDB Atlas delete events only return the `_id` by default, if the sync condition for the target database (e.g., MySQL) does not include `_id`, the delete operation may fail. In this case, you can first sync the source table **A** to an intermediate table **A_COPY** (using `_id` as the update condition and enabling this option), then create a new task to sync from **A_COPY** to the target table **B**. |
162+
163+
</TabItem>
164+
</Tabs>

0 commit comments

Comments
 (0)