Skip to content

Add Pipelines Show Command - #6033

Open
tanishgupta-db wants to merge 6 commits into
databricks:mainfrom
tanishgupta-db:preview-datasets
Open

Add Pipelines Show Command#6033
tanishgupta-db wants to merge 6 commits into
databricks:mainfrom
tanishgupta-db:preview-datasets

Conversation

@tanishgupta-db

@tanishgupta-db tanishgupta-db commented Jul 22, 2026

Copy link
Copy Markdown

Changes

Adds databricks pipelines show <catalog.schema.table> command: shows a table's columns and sample data (or --output json). Runs SELECT * FROM <table> LIMIT n on a SQL warehouse. The warehouse is taken from --warehouse-id, DATABRICKS_WAREHOUSE_ID (env set), the workspace default, or picked.

Also adds renderStaticWithTruncation in tableview for detecting terminal width and rendering a formatted table

Why

Users want a quick look at the data and schema of a dataset quickly without leaving the terminal. Mirrors the Catalog Explorer "Sample Data" experience for the CLI. Lives under the pipelines command group.

Tests

  • Unit tests and Acceptance tests

E2E testing:

Screen.Recording.2026-07-23.at.11.05.36.PM.mov

@tanishgupta-db
tanishgupta-db marked this pull request as ready for review July 23, 2026 04:47
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/pipelines/ - needs approval

12 files changed
Suggested: @kanterov
Also eligible: @jefferycheng1, @lennartkats-db

/cmd/pipelines/ - needs approval

7 files changed
Suggested: @kanterov
Also eligible: @jefferycheng1, @lennartkats-db

General files (require maintainer)

Files: NOTICE, go.mod
Based on git history:

  • @pietern -- recent work in cmd/pipelines/, ./

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.
See OWNERS for ownership rules.

@tanishgupta-db tanishgupta-db changed the title preview-datasets v1 Add Pipelines Show Command Jul 24, 2026
Comment thread acceptance/pipelines/show/basic/output.txt Outdated
Comment thread acceptance/pipelines/show/help/output.txt Outdated

=== json output

>>> [CLI] pipelines show main.default.people --output json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a user passes in a table name that already has backticks, like "main.default.people"?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the backticks are literal characters in the name (e.g. an MV whose catalog is named main). To address that correctly I double internal backticks, so such names are reachable.
if the user meant "main.default.people" and pre-quoted, they'll see a not-found error.

Comment thread libs/tableview/static.go Outdated
Comment thread libs/tableview/static.go Outdated
}

// fitColumns returns how many leading columns fit within width (<= 0 means unlimited) and whether any were dropped; at least one column is always shown.
func fitColumns(colWidth []int, width int) (shown int, cropped bool) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add acceptance tests which coveres this?

}

// writes rows as an indented JSON array of column-ordered objects.
func renderJSON(w io.Writer, columns []string, rows [][]string) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add an acceptance test for JSON output too?

return nil
}

// renderTable writes a width-aware text table: cells cap at maxColumnWidth, and a positive width that the table exceeds drops trailing columns behind a "..." marker.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to truncate? We don't really do this anywhere else, maybe bubbletea which we use underneath already handles this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think truncation is in general smth we should do and especially relevant in this case bc this command shows a preview of a users dataset, which could have a lot of columns or cells with very long values Ex. blobs.
Both of these cases would break any non-truncated rendering we did and make the output really hard to read.
Also bubbletea just renders an inifinite scrollable view horizontally so it doesn't do any truncation but keeps the output format clean and lets the user scroll to see the rest of their columns (this also isn't great for large cell values like blobs bc it prints the whole blob so the user has to scroll horizontally past the whole thing).

@github-actions

Copy link
Copy Markdown
Contributor

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 6033
  • Commit SHA: c8e4be98a3fe6fc259baef717144bb3d39c05c7d

Checks will be approved automatically on success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants