Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 5 additions & 135 deletions .github/workflows/spelling.yml
Original file line number Diff line number Diff line change
@@ -1,166 +1,36 @@
name: Check Spelling

# Comment management is handled through a secondary job, for details see:
# https://github.com/check-spelling/check-spelling/wiki/Feature%3A-Restricted-Permissions
#
# `jobs.comment-push` runs when a push is made to a repository and the `jobs.spelling` job needs to make a comment
# (in odd cases, it might actually run just to collapse a comment, but that's fairly rare)
# it needs `contents: write` in order to add a comment.
#
# `jobs.comment-pr` runs when a pull_request is made to a repository and the `jobs.spelling` job needs to make a comment
# or collapse a comment (in the case where it had previously made a comment and now no longer needs to show a comment)
# it needs `pull-requests: write` in order to manipulate those comments.

# Updating pull request branches is managed via comment handling.
# For details, see: https://github.com/check-spelling/check-spelling/wiki/Feature:-Update-expect-list
#
# These elements work together to make it happen:
#
# `on.issue_comment`
# This event listens to comments by users asking to update the metadata.
#
# `jobs.update`
# This job runs in response to an issue_comment and will push a new commit
# to update the spelling metadata.
#
# `with.experimental_apply_changes_via_bot`
# Tells the action to support and generate messages that enable it
# to make a commit to update the spelling metadata.
#
# `with.ssh_key`
# In order to trigger workflows when the commit is made, you can provide a
# secret (typically, a write-enabled github deploy key).
#
# For background, see: https://github.com/check-spelling/check-spelling/wiki/Feature:-Update-with-deploy-key

# Sarif reporting
#
# Access to Sarif reports is generally restricted (by GitHub) to members of the repository.
#
# Requires enabling `security-events: write`
# and configuring the action with `use_sarif: 1`
#
# For information on the feature, see: https://github.com/check-spelling/check-spelling/wiki/Feature:-Sarif-output

# Minimal workflow structure:
#
# on:
# push:
# ...
# pull_request_target:
# ...
# jobs:
# # you only want the spelling job, all others should be omitted
# spelling:
# # remove `security-events: write` and `use_sarif: 1`
# # remove `experimental_apply_changes_via_bot: 1`
# ... otherwise adjust the `with:` as you wish
# This workflow is intentionally manual-only so spelling does not block pushes
# or pull requests.

on:
push:
branches:
- "**"
tags-ignore:
- "**"
pull_request_target:
branches:
- "**"
types:
- 'opened'
- 'reopened'
- 'synchronize'
issue_comment:
types:
- 'created'
workflow_dispatch:

jobs:
spelling:
name: Check Spelling
permissions:
contents: read
pull-requests: read
actions: read
security-events: write
Comment thread
heywalter marked this conversation as resolved.
outputs:
followup: ${{ steps.spelling.outputs.followup }}
runs-on: ubuntu-latest
if: ${{ contains(github.event_name, 'pull_request') || github.event_name == 'push' }}
concurrency:
group: spelling-${{ github.event.pull_request.number || github.ref }}
group: spelling-${{ github.ref }}
# note: If you use only_check_changed_files, you do not want cancel-in-progress
cancel-in-progress: true
steps:
- name: check-spelling
id: spelling
uses: check-spelling/check-spelling@main
with:
suppress_push_for_open_pull_request: ${{ github.actor != 'dependabot[bot]' && 1 }}
checkout: true
check_file_names: 0
spell_check_this: check-spelling/spell-check-this@prerelease
post_comment: 0
use_magic_file: 0
report-timing: 1
warnings: bad-regex,binary-file,deprecated-feature,large-file,limited-references,no-newline-at-eof,noisy-file,non-alpha-in-dictionary,token-is-substring,unexpected-line-ending,whitespace-in-dictionary,minified-file,unsupported-configuration,no-files-to-check
experimental_apply_changes_via_bot: 1
use_sarif: ${{ (!github.event.pull_request || (github.event.pull_request.head.repo.full_name == github.repository)) && 1 }}
use_sarif: 1
extra_dictionary_limit: 20
extra_dictionaries:
cspell:software-terms/dict/softwareTerms.txt

comment-push:
name: Report (Push)
# If your workflow isn't running on push, you can remove this job
runs-on: ubuntu-latest
needs: spelling
permissions:
contents: write
if: (success() || failure()) && needs.spelling.outputs.followup && github.event_name == 'push'
steps:
- name: comment
uses: check-spelling/check-spelling@main
with:
checkout: true
spell_check_this: check-spelling/spell-check-this@prerelease
task: ${{ needs.spelling.outputs.followup }}

comment-pr:
name: Report (PR)
# If you workflow isn't running on pull_request*, you can remove this job
runs-on: ubuntu-latest
needs: spelling
permissions:
contents: read
pull-requests: write
if: (success() || failure()) && needs.spelling.outputs.followup && contains(github.event_name, 'pull_request')
steps:
- name: comment
uses: check-spelling/check-spelling@main
with:
checkout: true
spell_check_this: check-spelling/spell-check-this@prerelease
task: ${{ needs.spelling.outputs.followup }}
experimental_apply_changes_via_bot: 1

update:
name: Update PR
permissions:
contents: write
pull-requests: write
actions: read
runs-on: ubuntu-latest
if: ${{
github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@check-spelling-bot apply')
}}
concurrency:
group: spelling-update-${{ github.event.issue.number }}
cancel-in-progress: false
steps:
- name: apply spelling updates
uses: check-spelling/check-spelling@main
with:
experimental_apply_changes_via_bot: 1
checkout: true
ssh_key: "${{ secrets.CHECK_SPELLING }}"
4 changes: 2 additions & 2 deletions docs/case-practices/best-practice/heart-beat-task.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ TapData uses heartbeat tables to write timestamp information to the source datab

## Applicable Scenarios

* If the source database often has long periods without DML operations (such as data insertion), and you need to quickly determine the status of incremental tasks through the **Incremental Checkpoint** in the **Task List** to ensure the link status and incremental log acquisition are normal.
* If the source database often has long periods without DML operations (such as data insertion), and you need to quickly determine the status of incremental tasks through the **Last Event Time** in the **Task List** to ensure the link status and incremental log acquisition are normal.

![Incremental Checkpoint](../../images/incremental_check_point.png)
![Last Event Time](../../images/incremental_check_point.png)

* If the source database has many DML operations, it is necessary to prevent the incremental logs recorded by TapData from being archived and deleted, which would make it impossible to obtain the incremental checkpoint.

Expand Down
1 change: 1 addition & 0 deletions docs/connectors/warehouses-and-lake/paimon.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Add a [Type Modification Processor](../../data-transformation/process-node.md#ty
- To avoid write conflicts and reduce compaction pressure, disable multi-threaded writes in the target node and set batch size to 1,000 rows and timeout to 1,000 ms.
- Always define a primary key for efficient upserts and deletes; for large tables, use partitioning to speed up queries and writes.
- Paimon supports primary keys only (no secondary indexes) and does not allow runtime schema evolution.
- When Paimon is used as the target and soft delete is enabled, TapData converts `DELETE` operations into `UPDATE` operations with a delete marker. Because Paimon updates require complete row data, the source `DELETE` event must provide the full before image; otherwise, fields other than the primary key may be written as `null`. If the source database is MongoDB 6.0 or later, enable **[Document Pre-image](../on-prem-databases/mongodb.md#node-advanced-features)**. For other source databases, ensure that CDC logs contain the complete row data before the `DELETE` operation.

## Connect to Paimon

Expand Down
28 changes: 25 additions & 3 deletions docs/data-replication/monitor-task.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,37 @@
# Monitor Data Replication Task


Once the data replication task is started, the page will automatically redirect to the task monitoring page. From there, you can monitor the task's operation details, such as the status of the Agent, data synchronization progress, task progress, alert settings, and other relevant information.
After you start a data replication task, TapData automatically redirects you to the task monitoring page. You can use this page to check runtime details such as Agent status, synchronization status, task progress, and alert settings. You can also review the overall task status from the task list before opening a specific task for details.


:::tip

By clicking the **monitor** button on the task list page, you can access the monitoring page as well.
You can also open the monitoring page by clicking **Monitor** on the task list page.

:::


## Task List Metrics

On the **Data Replication** task list page, use filters such as **Task Status**, **Sync Type**, **Task Milestone**, **Agent Name**, and **Task Name** to find tasks. You can also click **Display Settings** to adjust the columns shown in the list. The following table describes the common fields:

| Field | Description |
| --- | --- |
| **Task Name** | The display name of the task. You can search tasks by this field. |
| **Task Status** | The current task status, such as Editing, Running, Stopped, or Failed. |
| **Sync Type** | The task synchronization type, such as full, incremental, or full + incremental. |
| **Task Milestone** | The current execution stage, which helps you determine whether the task is initializing, running full synchronization, running incremental synchronization, or processing another stage. |
| **Incremental Delay** | The time between when an event is generated at the source and when the task finishes processing it. Use this metric to identify incremental backlog. |
| **Last Event Time** | The latest source event time processed by the task. The closer this time is to the current time, the more up to date incremental processing is. |
| **Last Run Time** | The most recent time when the task was started. |
| **Create Time** | The time when the task was created. |
| **Update Time** | The most recent time when the task configuration or status was updated. |
| **Available Operation** | Operations available for the task, such as Start, Edit, Monitor, Reset, Copy, and Delete. The available operations depend on the task status. |

To view node-level metrics, run logs, alerts, or task settings, click **Monitor** on the right side of the target task to open the task details page.

## View Task Details

![](../images/monitor_copy_task_overview.png)


Expand Down Expand Up @@ -70,4 +93,3 @@ Click the ![](../images/task_setting_icon.png) icon at the top of the page, whic
* Task increment start notification
* Task stop alert
* Task increment delay alert

Loading
Loading