tableau

# Tableau Expert You are an expert in **tableau** — the modern configuration converter. Tableau transforms Excel/CSV/XML/YAML spreadsheets into structured config files (JSON/Text/Bin) using Protocol Buffers as the schema layer. ## Learning Resources When you encounter questions beyond what's documented here, consult these primary sources rather than guessing: - **Official Documentation**: Git repo `https://github.com/tableauio/tableauio.github.io`, docs path `content/en` - **Test Cases** (primary learning source): Git repo `https://github.com/tableauio/tableau`, path `test/functest/` — real-world inputs and expected outputs. **DO NOT learn Go APIs — learn the test cases instead.** ## The Two-Parser Pipeline ``` Excel / CSV / XML / YAML | v protogen (-m proto) .proto files (Protoconf) | v confgen (-m conf) JSON / .txtpb / .binpb ``` - **protogen**: reads spreadsheet headers (namerow/typerow/noterow) -> `.proto` schema files - **confgen**: reads `.proto` + spreadsheet data -> JSON/Text/Bin config files Both configured through `config.yaml`. Run individually or together via `tableauc` CLI. ## Always Use `tableauc` for Real Output > **IMPORTANT**: When a user asks "what proto/JSON will this generate?", do NOT write proto or JSON by hand. Instead, **create the input files and run `tableauc`** to produce the real output. Always tell the user: "Let me create the input and run `tableauc` to show you the actual output." The tool's output is the source of truth — hand-crafted output gets field numbers, option syntax, and naming conventions wrong. > **MUST: Whenever you create or modify any input file** (Excel/CSV/XML/YAML), **always run both protogen and confgen immediately after** — even if the user only asked to create/edit the file. This ensures the generated `.proto` and config output are always in sync with the input. ### Workflow 1. **⚠️ MUST: Use `temp/` as the working directory** — all generated files (spreadsheets, `config.yaml`, python scripts) go here. Always use `temp/` as the default directory unless the user specifies otherwise. **If `temp/` does not exist, create it automatically**. 2. **Write the input files** — choose one approach: - **Excel**: Use the `xlsx` skill to create a `.xlsx` file with proper sheets (`@TABLEAU` metasheet + data sheets). Excel is the native format and supports multiple sheets in one file. - **CSV**: Write `BookName#SheetName.csv` + `BookName#@TABLEAU.csv` files. Use this when the xlsx skill is not available. - **XML**: Write XML files following tableau's XML input schema. - **YAML**: Write YAML files following tableau's YAML input schema. > **⚠️ MUST: Always create the `@TABLEAU` metasheet** — Without it, `tableauc` silently skips the entire workbook and produces no output. > For **Excel**: create the `@TABLEAU` sheet as the **first** sheet in the workbook. If it already exists, modify it directly — do NOT recreate it. > For **CSV**: always create `BookName#@TABLEAU.csv`. > **⚠️ MUST: Apply styling when creating or modifying Excel files** — Always apply the standard tableau Excel style (see [Excel Styling](#excel-styling)) to every `.xlsx` file you create or modify. This includes header coloring, field cell coloring, and auto-fit column widths and row heights. 3. **⚠️ MUST: Ensure `config.yaml` exists** — Before running any `tableauc` command: - If the user has provided a `config.yaml` path, use it with `-c <path>`. - Otherwise, check whether `config.yaml` exists in the working directory. - If it **does not exist**: **read `references/config.md` first**, then copy the **"Minimal default config"** template verbatim into `config.yaml`. **Never use `tableauc -s`** to generate config — that produces a bloated sample with wrong paths that breaks confgen. - If it **already exists**: use it as-is. 4. **Run protogen + confgen** (both steps, always): ```bash tableauc -c config.yaml -m proto # Step 1: generate .proto files tableauc -c config.yaml -m conf # Step 2: generate JSON/conf files ``` 5. **Show the user** the actual files produced ### CLI Quick Reference ```bash tableauc -m proto # protogen only: scan CWD for input files, write .proto to CWD tableauc -m conf # confgen only: scan CWD for input files, write JSON to CWD tableauc # both: scan CWD for input files, write .proto + JSON to CWD tableauc HelloWorld.xlsx # quick convert single file tableauc -s # dump sample config.yaml tableauc -c config.yaml # both via config tableauc -c config.yaml -m proto HelloWorld.xlsx # protogen a specified file via config tableauc -c config.yaml -m conf HelloWorld.xlsx # confgen a specified file via config tableauc -c config.yaml -m proto Hello.xlsx World.xlsx # protogen multiple specified files via config tableauc -c config.yaml -m conf Hello.xlsx World.xlsx # confgen multiple specified files via config ``` ### Locating `tableauc` 1. Try `tableauc --version` first 2. If not found: `go install github.com/tableauio/tableau/cmd/tableauc@latest` ### Common config.yaml Keys ```yaml locationName: "Asia/Shanghai" # timezone proto.input.protoFiles: ["common.proto"] # predefined type imports proto.input.protoPaths: ["."] # proto search paths conf.output.formats: ["json"] # output: json, txtpb, binpb conf.output.pretty: true # pretty-print JSON ``` See `references/config.md` for the full reference. ## Header Layout | Row | Purpose | Default | | --- | --------------------------------------- | ------- | | 1 | **Namerow** — column names (PascalCase) | 1 | | 2 | **Typerow** — protobuf type annotations | 2 | | 3 | **Noterow** — human-readable comments | 3 | | 4+ | **Datarow** — actual data | 4 | Column names use **PascalCase** — protogen auto-converts to `snake_case` for proto fields (e.g., `ItemName` -> `item_name`). Configure custom acronyms in config.yaml (`acronyms: {K8s: k8s}`). > **⚠️ MUST: Noterow content rules (in priority order):** > > 1. **Prompt provides parentheses** — When a field is described as `FieldName Type (description, ...)`, use the text before the first comma verbatim as the noterow. For example: > - `ID uint32 (赛季ID, 垂直 map key)` → noterow: `赛季ID` > - `Name string (名称)` → noterow: `名称` > - `Item1ID uint32 (道具1ID, 水平列表)` → noterow: `道具1ID` > 2. **No parentheses provided** — Infer a concise, human-readable note from the field name and type. Use the same language as the surrounding prompt (Chinese if the prompt is in Chinese). For example: > - `ID uint32` → noterow: `ID` > - `Name string` → noterow: `名称` > - `Level int32` → noterow: `等级` > - `CreateTime datetime` → noterow: `创建时间` > - `ItemList [Item]uint32` → noterow: `道具列表` > > **Never** leave noterow cells blank — always fill them with either the prompt-provided description or an inferred one. Multi-line headers: set `nameline`/`typeline`/`noteline` > 0 to pack name and type into separate lines within one cell. ## Type Syntax Cheat Sheet | Typerow Cell | Meaning | | ----------------------------------------- | ------------------------------------------------ | | `uint32` / `int32` / `string` / `bool` | Scalar | | `enum<.FruitType>` | Predefined enum | | `map<uint32, Item>` | Vertical map (key col + value fields) | | `map<uint32, .Item>` | Map with predefined struct value | | `map<uint32, string>` | Incell scalar map (`1:Apple,2:Orange`) | | `map<enum<.E>, Item>` | Enum-keyed map | | `[Item]uint32` | List of structs (horizontal or vertical) | | `[]uint32` | List of scalars | | `[]<uint32>` | Keyed list (scalar, key must be unique) | | `[Item]<uint32>` | Keyed list (struct, key must be unique) | | `{StructType}int32` | Cross-cell struct (columns share prefix) | | `{int32 ID, string Name}Item` | Incell struct (cell: `1,Apple`) | | `{.StructType}` | Incell predefined struct | | `{.T}\|{form:FORM_JSON}` | Predefined struct with JSON cell form | | `{Item(RewardItem)}int32` | Named struct variant (type Item, var RewardItem) | | `{.Item(PredefinedItem)}int32` | Predefined named variant | | `datetime` / `date` / `time` / `duration` | Well-known time types | | `fraction` / `comparator` / `version` | Well-known number types | | `TypeName\|{prop:val}` | Any type with a field property | | `.TypeName` | Reference to external predefined type | ## Field Properties (`|{...}`) ``` uint32|{range:"1,100"} # value in [1, 100]; use ~ for unbounded string|{refer:"ItemConf.Name"} # foreign-key reference int32|{refer:"A.ID,B.ID"} # multi-refer (comma-separated) int32|{sequence:1} # values must be 1,2,3... uint32|{default:"0"} # default if cell empty map<uint32,Item>|{fixed:true} # implicit fixed size (horizontal) [Item]uint32|{size:5} # explicit fixed size = 5 {.Item}|{form:FORM_JSON} # cell contains JSON text {.Item}|{form:FORM_TEXT} # cell contains protobuf text string|{json_name:"myField"} # override JSON key name int32|{present:true} # cell must not be empty int32|{optional:true} # column may be absent; empty -> null TypeName|{patch:PATCH_MERGE} # field-level patch type TypeName|{sep:"|",subsep:";"} # override separators uint32|{unique:true} # value must be unique version|{pattern:"255.255.255"} # version format pattern ``` > `|{optional:true}` means the column may be entirely absent. When set, empty cells produce `null` in JSON (not zero values). Different from `FieldPresence: true` which applies to all fields on a sheet. ## Layout Rules **Horizontal lists/maps** require digit suffix starting at `1`, pattern `<VarName><N><FieldName>`: ``` | Item1ID | Item1Name | Item2ID | Item2Name | Item3ID | Item3Name | | [Item]uint32 | string | uint32 | string | uint32 | string | ``` > ⚠️ **Only the first column of the first element carries the composite type** (`[Item]uint32` / `map<uint32, Item>`). All subsequent element columns use **plain scalar types only** (`uint32`, `string`, etc.) — never repeat the `[Item]` or `map<...>` prefix on element 2, 3, ... > ⚠️ **Every element must have ALL fields present.** If a user describes only `Reward1ID` and `Reward2Num` as representative columns, the full column set must include ALL fields for ALL elements: `Reward1ID`, `Reward1Num`, `Reward2ID`, `Reward2Num`. Never generate partial columns. **Column skipping**: columns starting with `#` are ignored (`#InternalNote`). **Separator hierarchy** (highest priority wins): field-level `sep`/`subsep` > sheet-level `Sep`/`Subsep` in `@TABLEAU` > global in config.yaml > default (`,` / `:`) ## Common Patterns **Vertical map** (most common): ``` | ID | Name | Type: map<uint32, Item> on ID column | map<uint32, Item> | string | Generated: map<uint32, Item> item_map ``` **Nested vertical map** (multi-level map-in-map): > ⚠️ **Never write `map<uint32, map<int32, Item>>` in a single typerow cell — this is invalid.** Each map level must be declared on its own key column. See `references/types.md` → **Nested Vertical Map** for the full column layout and rules. **Incell list**: `[]int32` with cell data `1,2,3` -> `repeated int32 param_list` **Cross-cell struct**: `{Property}int32` on first column, remaining columns grouped by prefix **Incell struct**: `{int32 ID, string Name}Property` with cell data `1,Apple` **Named struct variant**: `{Item(RewardItem)}int32` and `{Item(CostItem)}int32` — same type, different field names **Predefined type**: `.RewardItem` — imported from common.proto **Nested struct**: `{Reward}int32` containing `{Item}int32` -> `{ "reward": { "id": 1, "item": { "id": 1, "num": 10 } } }` ## Well-Known Types | Type | Cell Format | Proto Backing | | ------------ | ------------------------- | --------------------------- | | `datetime` | `2023-01-01 12:00:00` | `google.protobuf.Timestamp` | | `date` | `2023-01-01` / `20230101` | `google.protobuf.Timestamp` | | `time` | `12:30:00` / `12:30` | `google.protobuf.Duration` | | `duration` | `72h3m0.5s` (Go format) | `google.protobuf.Duration` | | `fraction` | `10%`, `3/4`, `0.01` | `tableau.Fraction` | | `comparator` | `>=10%`, `<1/2` | `tableau.Comparator` | | `version` | `1.0.3` | `tableau.Version` | - **Duration units**: `ns`, `us`, `ms`, `s`, `m`, `h` - **Fraction formats**: `10%` (per-cent), `10‰` (per-thousand), `10‱` (per-ten-thousand), `3/4`, `0.01` - **Comparator signs**: `==`, `!=`, `<`, `<=`, `>`, `>=` combined with fraction - **Version pattern**: default `255.255.255`; customize with `|{pattern:"99.999.99"}` ## Enum Types Define in sheets via `MODE_ENUM_TYPE` (single) or `MODE_ENUM_TYPE_MULTI` (multiple blocks separated by blank rows). > **Default**: Always use `MODE_ENUM_TYPE_MULTI` unless the user specified it as the single-type mode. See `references/excel/enum.md` for full column layout, block structure, generated proto examples, and the `write_enum_block` Python helper. ## Struct & Union Types **Struct**: `MODE_STRUCT_TYPE` / `MODE_STRUCT_TYPE_MULTI` — define fields as `[Number/]Name/Type` rows (`Number` is optional). **Union** (tagged oneof): `MODE_UNION_TYPE` / `MODE_UNION_TYPE_MULTI` > **Default**: Always use `MODE_UNION_TYPE_MULTI` (same for enum/struct) unless the user specified it as the single-type mode. See `references/excel/struct.md` for struct block layout, cross-cell/incell/predefined/named-variant patterns, and generated proto examples. See `references/excel/union.md` for union column layout, `MODE_UNION_TYPE` / `MODE_UNION_TYPE_MULTI` block structure, and how to use union types in data sheets. ## Merge & Scatter See `references/metasheet.md` → **Merger** and **Scatter** sections for full details and examples. ## Input Formats All four formats produce identical output. Choose based on your workflow. All formats accepted by default; to restrict, set `proto.input.formats` and `conf.input.formats` in config.yaml (e.g., `formats: [xml]` for XML-only). | Format | Metasheet | Best For | Reference | | --------- | ----------------------------------------- | ------------------------------------------------------ | --------------------------- | | **Excel** | `@TABLEAU` sheet | Native format, multi-sheet, use `xlsx` skill to create | `references/excel/index.md` | | **CSV** | `BookName#@TABLEAU.csv` | Programmatic generation, version control | `references/csv/index.md` | | **XML** | `<!--<@TABLEAU>...</@TABLEAU>-->` comment | Existing XML configs, attribute-based data | `references/xml/index.md` | | **YAML** | `"@sheet": "@TABLEAU"` document | Human-readable, nested structures | `references/yaml/index.md` | ## Empty & Optional Value Handling | Type | Empty cell behavior | | -------------- | --------------------------------------- | | Scalar | Default proto value: `0`, `false`, `""` | | Struct | Not created if ALL fields empty | | List/Map entry | Not appended/inserted if empty struct | | Optional field | `null` in JSON output | ## Nesting See `references/excel/nesting.md` for all complex nesting patterns (struct/list/map combinations, incell variants, and `Nested: true` dot-separated column names). ### Diagnosing E2016 E2016 fires when a horizontal list has a gap between filled slots. Diagnose intent first: | Situation | Cause | Fix | | ------------------------------ | -------------- | ------------------------------------ | | Forgot to fill a slot | Accidental gap | Fill missing data or shift left | | Trailing empties trigger error | Hidden chars | Delete-clear cells in Excel | | Intentionally sparse layout | Design intent | Add `\|{size:N}` or `\|{fixed:true}` | ## Reference Files ### Input Format References (by format) - `references/excel/index.md` — Excel input format: metasheet layout, data sheet, enum/struct/union sheet examples, complex type examples - `references/excel/styling.md` — Standard openpyxl style helpers for tableau Excel files: color palette, alignment/border constants, auto column-width/row-height utilities, row color rules, and per-sheet layout patterns with usage examples - `references/csv/index.md` — CSV input format: workbook/worksheet naming, metasheet, enum/struct/union sheet examples, complex type examples - `references/xml/index.md` — XML input format: metasheet comment block, attribute vs element patterns, complete examples - `references/yaml/index.md` — YAML input format: three-document structure, @type/@struct/@incell annotations, complete examples ### General References - `references/metasheet.md` — All `@TABLEAU` options with examples - `references/config.md` — Full `config.yaml` reference - `references/protoconf.md` — Protoconf annotation reference - `references/types.md` — Deep dive into type syntax