๊ธฐ์กด ์ŠคํŠธ๋ฆฌ๋ฐ API ์‚ฌ์šฉ

์ด ๋ฌธ์„œ์—์„œ๋Š” ๊ธฐ์กด tabledata.insertAll ๋ฉ”์„œ๋“œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐ์ดํ„ฐ๋ฅผ BigQuery๋กœ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค.

์ƒˆ ํ”„๋กœ์ ํŠธ์˜ ๊ฒฝ์šฐ tabledata.insertAll ๋ฉ”์„œ๋“œ ๋Œ€์‹  BigQuery Storage Write API๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค. Storage Write API๋Š” ์ •ํ™•ํžˆ 1ํšŒ ์ „์†ก ์‹œ๋งจํ‹ฑ์Šค๋ฅผ ๋น„๋กฏํ•ด ๋” ์ €๋ ดํ•œ ๊ฐ€๊ฒฉ๊ณผ ๋” ๊ฐ•๋ ฅํ•œ ๊ธฐ๋Šฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ์กด ํ”„๋กœ์ ํŠธ๋ฅผ tabledata.insertAll ๋ฉ”์„œ๋“œ์—์„œ Storage Write API๋กœ ๋งˆ์ด๊ทธ๋ ˆ์ด์…˜ํ•˜๋Š” ๊ฒฝ์šฐ ๊ธฐ๋ณธ ์ŠคํŠธ๋ฆผ์„ ์„ ํƒํ•˜๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค. tabledata.insertAll ๋ฉ”์„œ๋“œ๋Š” ๊ณ„์†ํ•ด์„œ ์™„๋ฒฝํžˆ ์ง€์›๋ฉ๋‹ˆ๋‹ค.

์‹œ์ž‘ํ•˜๊ธฐ ์ „์—

  1. ๋Œ€์ƒ ํ…Œ์ด๋ธ”์ด ํฌํ•จ๋œ ๋ฐ์ดํ„ฐ ์„ธํŠธ์— ๋Œ€ํ•œ ์“ฐ๊ธฐ ์•ก์„ธ์Šค ๊ถŒํ•œ์ด ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ๋ฅผ ์ œ์™ธํ•˜๊ณ  ๋ฐ์ดํ„ฐ๋ฅผ ์“ฐ๊ธฐ ์ „์— ํ…Œ์ด๋ธ”์ด ์กด์žฌํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜์—ฌ ์ž๋™์œผ๋กœ ํ…Œ์ด๋ธ” ๋งŒ๋“ค๊ธฐ๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

  2. ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ์˜ ํ• ๋‹น๋Ÿ‰ ์ •์ฑ…์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

  3. Make sure that billing is enabled for your Google Cloud project.

  4. ๋ฌด๋ฃŒ ๋“ฑ๊ธ‰์œผ๋กœ๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค. ๊ฒฐ์ œ๋ฅผ ์‚ฌ์šฉ ์„ค์ •ํ•˜์ง€ ์•Š๊ณ  ์ŠคํŠธ๋ฆฌ๋ฐ์„ ์‚ฌ์šฉํ•˜๋ฉด BigQuery: Streaming insert is not allowed in the free tier. ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.

  5. ์‚ฌ์šฉ์ž์—๊ฒŒ ์ด ๋ฌธ์„œ์˜ ๊ฐ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐ ํ•„์š”ํ•œ ๊ถŒํ•œ์„ ๋ถ€์—ฌํ•˜๋Š” Identity and Access Management(IAM) ์—ญํ• ์„ ๋ถ€์—ฌํ•ฉ๋‹ˆ๋‹ค.

ํ•„์ˆ˜ ๊ถŒํ•œ

BigQuery๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋ ค๋ฉด ๋‹ค์Œ IAM ๊ถŒํ•œ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

  • bigquery.tables.updateData(ํ…Œ์ด๋ธ”์— ๋ฐ์ดํ„ฐ ์‚ฝ์ž… ๊ฐ€๋Šฅ)
  • bigquery.tables.get(ํ…Œ์ด๋ธ” ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ๋ฅผ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ์Œ)
  • bigquery.datasets.get(๋ฐ์ดํ„ฐ ์„ธํŠธ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ๋ฅผ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ์Œ)
  • bigquery.tables.create(ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜์—ฌ ํ…Œ์ด๋ธ”์„ ์ž๋™์œผ๋กœ ๋งŒ๋“œ๋Š” ๊ฒฝ์šฐ ํ•„์š”)

์‚ฌ์ „ ์ •์˜๋œ ๋‹ค์Œ ๊ฐ IAM ์—ญํ• ์—๋Š” BigQuery๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๋ฐ ํ•„์š”ํ•œ ๊ถŒํ•œ์ด ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.

  • roles/bigquery.dataEditor
  • roles/bigquery.dataOwner
  • roles/bigquery.admin

BigQuery์˜ IAM ์—ญํ• ๊ณผ ๊ถŒํ•œ์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ์‚ฌ์ „ ์ •์˜๋œ ์—ญํ•  ๋ฐ ๊ถŒํ•œ์„ ์ฐธ์กฐํ•˜์„ธ์š”.

BigQuery์— ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ

C#

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ C# ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery C# API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.


using Google.Cloud.BigQuery.V2;

public class BigQueryTableInsertRows
{
    public void TableInsertRows(
        string projectId = "your-project-id",
        string datasetId = "your_dataset_id",
        string tableId = "your_table_id"
    )
    {
        BigQueryClient client = BigQueryClient.Create(projectId);
        BigQueryInsertRow[] rows = new BigQueryInsertRow[]
        {
            // The insert ID is optional, but can avoid duplicate data
            // when retrying inserts.
            new BigQueryInsertRow(insertId: "row1") {
                { "name", "Washington" },
                { "post_abbr", "WA" }
            },
            new BigQueryInsertRow(insertId: "row2") {
                { "name", "Colorado" },
                { "post_abbr", "CO" }
            }
        };
        client.InsertRows(datasetId, tableId, rows);
    }
}

Go

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Go ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Go API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

import (
	"context"
	"fmt"

	"cloud.google.com/go/bigquery"
)

// Item represents a row item.
type Item struct {
	Name string
	Age  int
}

// Save implements the ValueSaver interface.
// This example disables best-effort de-duplication, which allows for higher throughput.
func (i *Item) Save() (map[string]bigquery.Value, string, error) {
	return map[string]bigquery.Value{
		"full_name": i.Name,
		"age":       i.Age,
	}, bigquery.NoDedupeID, nil
}

// insertRows demonstrates inserting data into a table using the streaming insert mechanism.
func insertRows(projectID, datasetID, tableID string) error {
	// projectID := "my-project-id"
	// datasetID := "mydataset"
	// tableID := "mytable"
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, projectID)
	if err != nil {
		return fmt.Errorf("bigquery.NewClient: %w", err)
	}
	defer client.Close()

	inserter := client.Dataset(datasetID).Table(tableID).Inserter()
	items := []*Item{
		// Item implements the ValueSaver interface.
		{Name: "Phred Phlyntstone", Age: 32},
		{Name: "Wylma Phlyntstone", Age: 29},
	}
	if err := inserter.Put(ctx, items); err != nil {
		return err
	}
	return nil
}

Java

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Java ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Java API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryError;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.InsertAllRequest;
import com.google.cloud.bigquery.InsertAllResponse;
import com.google.cloud.bigquery.TableId;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

// Sample to inserting rows into a table without running a load job.
public class TableInsertRows {

  public static void main(String[] args) {
    // TODO(developer): Replace these variables before running the sample.
    String datasetName = "MY_DATASET_NAME";
    String tableName = "MY_TABLE_NAME";
    // Create a row to insert
    Map<String, Object> rowContent = new HashMap<>();
    rowContent.put("booleanField", true);
    rowContent.put("numericField", "3.14");
    // TODO(developer): Replace the row id with a unique value for each row.
    String rowId = "ROW_ID";
    tableInsertRows(datasetName, tableName, rowId, rowContent);
  }

  public static void tableInsertRows(
      String datasetName, String tableName, String rowId, Map<String, Object> rowContent) {
    try {
      // Initialize client that will be used to send requests. This client only needs to be created
      // once, and can be reused for multiple requests.
      BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

      // Get table
      TableId tableId = TableId.of(datasetName, tableName);

      // Inserts rowContent into datasetName:tableId.
      InsertAllResponse response =
          bigquery.insertAll(
              InsertAllRequest.newBuilder(tableId)
                  // More rows can be added in the same RPC by invoking .addRow() on the builder.
                  // You can omit the unique row ids to disable de-duplication.
                  .addRow(rowId, rowContent)
                  .build());

      if (response.hasErrors()) {
        // If any of the insertions failed, this lets you inspect the errors
        for (Map.Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
          System.out.println("Response error: \n" + entry.getValue());
        }
      }
      System.out.println("Rows successfully inserted into table");
    } catch (BigQueryException e) {
      System.out.println("Insert operation not performed \n" + e.toString());
    }
  }
}

Node.js

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Node.js ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Node.js API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

// Import the Google Cloud client library
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();

async function insertRowsAsStream() {
  // Inserts the JSON objects into my_dataset:my_table.

  /**
   * TODO(developer): Uncomment the following lines before running the sample.
   */
  // const datasetId = 'my_dataset';
  // const tableId = 'my_table';
  const rows = [
    {name: 'Tom', age: 30},
    {name: 'Jane', age: 32},
  ];

  // Insert data into a table
  await bigquery.dataset(datasetId).table(tableId).insert(rows);
  console.log(`Inserted ${rows.length} rows`);
}

PHP

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ PHP ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery PHP API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

use Google\Cloud\BigQuery\BigQueryClient;

/**
 * Stream data into bigquery
 *
 * @param string $projectId The project Id of your Google Cloud Project.
 * @param string $datasetId The BigQuery dataset ID.
 * @param string $tableId The BigQuery table ID.
 * @param string $data Json encoded data For eg,
 *    $data = json_encode([
 *       "field1" => "value1",
 *       "field2" => "value2",
 *    ]);
 */
function stream_row(
    string $projectId,
    string $datasetId,
    string $tableId,
    string $data
): void {
    // instantiate the bigquery table service
    $bigQuery = new BigQueryClient([
      'projectId' => $projectId,
    ]);
    $dataset = $bigQuery->dataset($datasetId);
    $table = $dataset->table($tableId);

    $data = json_decode($data, true);
    $insertResponse = $table->insertRows([
      ['data' => $data],
      // additional rows can go here
    ]);
    if ($insertResponse->isSuccessful()) {
        print('Data streamed into BigQuery successfully' . PHP_EOL);
    } else {
        foreach ($insertResponse->failedRows() as $row) {
            foreach ($row['errors'] as $error) {
                printf('%s: %s' . PHP_EOL, $error['reason'], $error['message']);
            }
        }
    }
}

Python

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Python ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Python API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set table_id to the ID of table to append to.
# table_id = "your-project.your_dataset.your_table"

rows_to_insert = [
    {"full_name": "Phred Phlyntstone", "age": 32},
    {"full_name": "Wylma Phlyntstone", "age": 29},
]

errors = client.insert_rows_json(table_id, rows_to_insert)  # Make an API request.
if errors == []:
    print("New rows have been added.")
else:
    print("Encountered errors while inserting rows: {}".format(errors))

Ruby

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Ruby ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Ruby API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

require "google/cloud/bigquery"

def table_insert_rows dataset_id = "your_dataset_id", table_id = "your_table_id"
  bigquery = Google::Cloud::Bigquery.new
  dataset  = bigquery.dataset dataset_id
  table    = dataset.table table_id

  row_data = [
    { name: "Alice", value: 5  },
    { name: "Bob",   value: 10 }
  ]
  response = table.insert row_data

  if response.success?
    puts "Inserted rows successfully"
  else
    puts "Failed to insert #{response.error_rows.count} rows"
  end
end

ํ–‰์„ ์‚ฝ์ž…ํ•  ๋•Œ insertID ํ•„๋“œ๋ฅผ ์ฑ„์šธ ํ•„์š”๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค. ๋‹ค์Œ ์˜ˆ์‹œ๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ ์‹œ ๊ฐ ํ–‰์— insertID๋ฅผ ์ „์†กํ•˜์ง€ ์•Š๋Š” ๋ฐฉ๋ฒ•์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.

Java

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Java ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Java API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryError;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.InsertAllRequest;
import com.google.cloud.bigquery.InsertAllResponse;
import com.google.cloud.bigquery.TableId;
import com.google.common.collect.ImmutableList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

// Sample to insert rows without row ids in a table
public class TableInsertRowsWithoutRowIds {

  public static void main(String[] args) {
    // TODO(developer): Replace these variables before running the sample.
    String datasetName = "MY_DATASET_NAME";
    String tableName = "MY_TABLE_NAME";
    tableInsertRowsWithoutRowIds(datasetName, tableName);
  }

  public static void tableInsertRowsWithoutRowIds(String datasetName, String tableName) {
    try {
      // Initialize client that will be used to send requests. This client only needs to be created
      // once, and can be reused for multiple requests.
      final BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
      // Create rows to insert
      Map<String, Object> rowContent1 = new HashMap<>();
      rowContent1.put("stringField", "Phred Phlyntstone");
      rowContent1.put("numericField", 32);
      Map<String, Object> rowContent2 = new HashMap<>();
      rowContent2.put("stringField", "Wylma Phlyntstone");
      rowContent2.put("numericField", 29);
      InsertAllResponse response =
          bigquery.insertAll(
              InsertAllRequest.newBuilder(TableId.of(datasetName, tableName))
                  // No row ids disable de-duplication, and also disable the retries in the Java
                  // library.
                  .setRows(
                      ImmutableList.of(
                          InsertAllRequest.RowToInsert.of(rowContent1),
                          InsertAllRequest.RowToInsert.of(rowContent2)))
                  .build());

      if (response.hasErrors()) {
        // If any of the insertions failed, this lets you inspect the errors
        for (Map.Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
          System.out.println("Response error: \n" + entry.getValue());
        }
      }
      System.out.println("Rows successfully inserted into table without row ids");
    } catch (BigQueryException e) {
      System.out.println("Insert operation not performed \n" + e.toString());
    }
  }
}

Python

์ด ์ƒ˜ํ”Œ์„ ์‚ฌ์šฉํ•ด ๋ณด๊ธฐ ์ „์— BigQuery ๋น ๋ฅธ ์‹œ์ž‘: ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ์˜ Python ์„ค์ • ์•ˆ๋‚ด๋ฅผ ๋”ฐ๋ฅด์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ BigQuery Python API ์ฐธ๊ณ  ๋ฌธ์„œ๋ฅผ ํ™•์ธํ•˜์„ธ์š”.

BigQuery์— ์ธ์ฆํ•˜๋ ค๋ฉด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ธฐ๋ณธ ์‚ฌ์šฉ์ž ์ธ์ฆ ์ •๋ณด๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ํด๋ผ์ด์–ธํŠธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์˜ ์ธ์ฆ ์„ค์ •์„ ์ฐธ์กฐํ•˜์„ธ์š”.

from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set table_id to the ID of table to append to.
# table_id = "your-project.your_dataset.your_table"

rows_to_insert = [
    {"full_name": "Phred Phlyntstone", "age": 32},
    {"full_name": "Wylma Phlyntstone", "age": 29},
]

errors = client.insert_rows_json(
    table_id, rows_to_insert, row_ids=[None] * len(rows_to_insert)
)  # Make an API request.
if errors == []:
    print("New rows have been added.")
else:
    print("Encountered errors while inserting rows: {}".format(errors))

๋‚ ์งœ ๋ฐ ์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ „์†ก

๋‚ ์งœ ๋ฐ ์‹œ๊ฐ„ ํ•„๋“œ์˜ ๊ฒฝ์šฐ ๋‹ค์Œ๊ณผ ๊ฐ™์ด tabledata.insertAll ๋ฉ”์„œ๋“œ๋กœ ๋ฐ์ดํ„ฐ ํ˜•์‹์„ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค.

์œ ํ˜• ํ˜•์‹
DATE "YYYY-MM-DD" ํ˜•์‹์˜ ๋ฌธ์ž์—ด
DATETIME "YYYY-MM-DD [HH:MM:SS]" ํ˜•์‹์˜ ๋ฌธ์ž์—ด
TIME "HH:MM:SS" ํ˜•์‹์˜ ๋ฌธ์ž์—ด
TIMESTAMP 1970-01-01(์œ ๋‹‰์Šค ์‹œ๊ฐ„) ์ดํ›„์˜ ์ดˆ ์ˆ˜ ๋˜๋Š” "YYYY-MM-DD HH:MM[:SS]" ํ˜•์‹์˜ ๋ฌธ์ž์—ด

๋ฒ”์œ„ ๋ฐ์ดํ„ฐ ์ „์†ก

RANGE<T> ์œ ํ˜• ํ•„๋“œ์˜ ๊ฒฝ์šฐ start ๋ฐ end ํ•„๋“œ๋ฅผ ์‚ฌ์šฉํ•ด์„œ tabledata.insertAll ๋ฉ”์„œ๋“œ์˜ ๋ฐ์ดํ„ฐ ํ˜•์‹์„ JSON ๊ฐ์ฒด๋กœ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค. start ๋ฐ end ํ•„๋“œ์˜ ๋ˆ„๋ฝ๋œ ๊ฐ’์ด๋‚˜ NULL ๊ฐ’์€ ๋ฐ”์ธ๋”ฉ๋˜์ง€ ์•Š์€ ๊ฒฝ๊ณ„๋ฅผ ๋‚˜ํƒ€๋ƒ…๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ํ•„๋“œ๋Š” T ์œ ํ˜•์˜ ๋™์ผํ•œ ์ง€์›๋˜๋Š” JSON ํ˜•์‹์„ ํฌํ•จํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ T๋Š” DATE, DATETIME, TIMESTAMP ์ค‘ ํ•˜๋‚˜์ผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋‹ค์Œ ์˜ˆ์‹œ์—์„œ f_range_date ํ•„๋“œ๋Š” ํ…Œ์ด๋ธ”์˜ RANGE<DATE> ์—ด์„ ๋‚˜ํƒ€๋ƒ…๋‹ˆ๋‹ค. tabledata.insertAll API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ด ์—ด์— ํ–‰์ด ์‚ฝ์ž…๋ฉ๋‹ˆ๋‹ค.

{
    "f_range_date": {
        "start": "1970-01-02",
        "end": null
    }
}

์ŠคํŠธ๋ฆผ ๋ฐ์ดํ„ฐ ๊ฐ€์šฉ์„ฑ

BigQuery๊ฐ€ tabledata.insertAll ์š”์ฒญ์„ ์„ฑ๊ณต์ ์œผ๋กœ ํ™•์ธํ•œ ์งํ›„์— GoogleSQL ์ฟผ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์‹ค์‹œ๊ฐ„ ๋ถ„์„์— ๋ฐ์ดํ„ฐ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ตœ๊ทผ์— ์ˆ˜์ง‘ ์‹œ๊ฐ„์œผ๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”์— ์ŠคํŠธ๋ฆฌ๋ฐ๋œ ํ–‰์—๋Š” _PARTITIONTIME ์œ ์‚ฌ ์—ด์— NULL ๊ฐ’์ด ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ํ–‰์˜ ๊ฒฝ์šฐ BigQuery๋Š” ์ผ๋ฐ˜์ ์œผ๋กœ ๋ช‡ ๋ถ„ ๋‚ด์— ๋ฐฑ๊ทธ๋ผ์šด๋“œ์—์„œ PARTITIONTIME ์—ด์˜ ์ตœ์ข… NULL์ด ์•„๋‹Œ ๊ฐ’์„ ํ• ๋‹นํ•ฉ๋‹ˆ๋‹ค. ๋“œ๋ฌผ์ง€๋งŒ ์ตœ๋Œ€ 90๋ถ„์ด ์†Œ์š”๋  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค.

์ตœ๊ทผ์— ์ŠคํŠธ๋ฆฌ๋ฐ๋œ ์ผ๋ถ€ ํ–‰์„ ์ผ๋ฐ˜์ ์œผ๋กœ ๋ช‡ ๋ถ„ ๋™์•ˆ ํ…Œ์ด๋ธ” ๋ณต์‚ฌ์— ์‚ฌ์šฉํ•˜์ง€ ๋ชปํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋“œ๋ฌผ์ง€๋งŒ ์ตœ๋Œ€ 90๋ถ„์ด ์†Œ์š”๋  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ๋ฅผ ํ…Œ์ด๋ธ” ๋ณต์‚ฌ์— ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š”์ง€ ์—ฌ๋ถ€๋ฅผ ๋ณด๋ ค๋ฉด streamingBuffer ์„น์…˜์˜ tables.get ์‘๋‹ต์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค. streamingBuffer ์„น์…˜์ด ์—†์œผ๋ฉด ๋ฐ์ดํ„ฐ๋ฅผ ๋ณต์‚ฌ์— ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. streamingBuffer.oldestEntryTime ํ•„๋“œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ์˜ ๋ ˆ์ฝ”๋“œ ๊ธฐ๊ฐ„์„ ์‹๋ณ„ํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค.

์ตœ์„ ํ˜• ์ค‘๋ณต ์‚ญ์ œ

์‚ฝ์ž…๋œ ํ–‰์— insertId์„ ์ž…๋ ฅํ•˜๋ฉด BigQuery๋Š” ์ด ID๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ตœ๋Œ€ 1๋ถ„ ๋™์•ˆ ์ตœ์„ ํ˜• ์ค‘๋ณต ์‚ญ์ œ๋ฅผ ์ง€์›ํ•ฉ๋‹ˆ๋‹ค. ์ฆ‰, ํ•ด๋‹น ๊ธฐ๊ฐ„ ๋‚ด์— ๊ฐ™์€ ํ…Œ์ด๋ธ” ๋‚ด์—์„œ ๊ฐ™์€ insertId๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋™์ผํ•œ ํ–‰์„ ๋‘ ๋ฒˆ ๋„˜๊ฒŒ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๊ฒฝ์šฐ BigQuery๊ฐ€ ํ•ด๋‹น ํ–‰์˜ ๋‹ค์ค‘ ์ผ์น˜ ํ•ญ๋ชฉ์„ ์ค‘๋ณต ์‚ญ์ œํ•˜๊ณ  ํ•ด๋‹น ์ผ์น˜ ํ•ญ๋ชฉ ์ค‘ ํ•˜๋‚˜๋งŒ ์ €์žฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์‹œ์Šคํ…œ์—์„œ๋Š” ๋™์ผํ•œ insertId๊ฐ€ ์ œ๊ณต๋œ ํ–‰๋„ ๋™์ผํ•  ๊ฒƒ์œผ๋กœ ์˜ˆ์ƒํ•ฉ๋‹ˆ๋‹ค. ๋‘ ํ–‰์˜ insertId๊ฐ€ ๊ฐ™๋‹ค๋ฉด BigQuery๊ฐ€ ๋ณด์กดํ•˜๋Š” ํ–‰์€ ํ™•์ •๋˜์–ด ์žˆ์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

์ค‘๋ณต ์‚ญ์ œ๋Š” ์ผ๋ฐ˜์ ์œผ๋กœ ์‹œ์Šคํ…œ๊ณผ BigQuery ์‚ฌ์ด์˜ ๋„คํŠธ์›Œํฌ ์˜ค๋ฅ˜ ๋˜๋Š” BigQuery ๋‚ด๋ถ€ ์˜ค๋ฅ˜์™€ ๊ฐ™์€ ํŠน์ • ์˜ค๋ฅ˜ ์ƒํƒœ์—์„œ๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์˜ ์ƒํƒœ๋ฅผ ํ™•์ธํ•  ๋ฐฉ๋ฒ•์ด ์—†๋Š” ๋ฐฐํฌ๋œ ์‹œ์Šคํ…œ์—์„œ ์‹œ๋‚˜๋ฆฌ์˜ค๋ฅผ ์žฌ์‹œ๋„ํ•˜๋Š” ๊ฒƒ์„ ์˜๋ฏธํ•ฉ๋‹ˆ๋‹ค. ์‚ฝ์ž…์„ ์žฌ์‹œ๋„ํ•˜๋Š” ๊ฒฝ์šฐ BigQuery๊ฐ€ ๋ฐ์ดํ„ฐ ์ค‘๋ณต ์‚ญ์ œ๋ฅผ ์‹œ๋„ํ•  ์ˆ˜ ์žˆ๋„๋ก ๋™์ผํ•œ ํ–‰ ์กฐํ•ฉ์— ๋™์ผํ•œ insertId๋ฅผ ์‚ฌ์šฉํ•˜์„ธ์š”. ์ž์„ธํ•œ ๋‚ด์šฉ์€ ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž… ๋ฌธ์ œ ํ•ด๊ฒฐ์„ ์ฐธ์กฐํ•˜์„ธ์š”.

BigQuery์—์„œ ์ œ๊ณตํ•˜๋Š” ์ค‘๋ณต ์‚ญ์ œ๋Š” ์ตœ์„ ์˜ ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. ๋˜ํ•œ ๋ฐ์ดํ„ฐ์— ์ค‘๋ณต์ด ์—†๋„๋ก ๋ณด์žฅํ•˜๋Š” ๋ฉ”์ปค๋‹ˆ์ฆ˜์œผ๋กœ ์˜์กดํ•ด ์‚ฌ์šฉํ•ด์„œ๋Š” ์•ˆ ๋ฉ๋‹ˆ๋‹ค. ๋˜ํ•œ BigQuery๋Š” ๋ฐ์ดํ„ฐ์˜ ์‹ ๋ขฐ์„ฑ๊ณผ ๊ฐ€์šฉ์„ฑ์„ ๋ณด์žฅํ•˜๊ธฐ ์œ„ํ•ด ์–ธ์ œ๋“ ์ง€ ์ตœ์„ ์˜ ์ค‘๋ณต ์‚ญ์ œ์˜ ํ’ˆ์งˆ์„ ์ €ํ•˜์‹œํ‚ฌ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ์˜ ์ค‘๋ณต ์‚ญ์ œ ์š”๊ตฌ์‚ฌํ•ญ์ด ์—„๊ฒฉํ•œ ๊ฒฝ์šฐ ํŠธ๋žœ์žญ์…˜์„ ์ง€์›ํ•˜๋Š” ๋Œ€์ฒด ์„œ๋น„์Šค๋กœ Google Cloud Datastore๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ตœ์„ ํ˜• ์ค‘๋ณต ์‚ญ์ œ ์‚ฌ์šฉ ์ค‘์ง€

์‚ฝ์ž…๋œ ๊ฐ ํ–‰์— ๋Œ€ํ•ด insertId ํ•„๋“œ๋ฅผ ์ž…๋ ฅํ•˜์ง€ ์•Š์Œ์œผ๋กœ์จ ์ตœ์„ ํ˜• ์ค‘๋ณต ์‚ญ์ œ๋ฅผ ์‚ฌ์šฉ ์ค‘์ง€ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ ์‚ฝ์ž…์— ๊ถŒ์žฅ๋˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค.

Apache Beam ๋ฐ Dataflow

Apache Beam์˜ Java์šฉ BigQuery I/O ์ปค๋„ฅํ„ฐ๋ฅผ ์‚ฌ์šฉํ•  ๋•Œ ์ตœ์„ ์˜ ์ค‘๋ณต ์ œ๊ฑฐ๋ฅผ ์‚ฌ์šฉ ์ค‘์ง€ํ•˜๋ ค๋ฉด ignoreInsertIds() ๋ฉ”์„œ๋“œ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

์ˆ˜๋™์œผ๋กœ ์ค‘๋ณต ์ œ๊ฑฐ

์ŠคํŠธ๋ฆฌ๋ฐ์„ ์™„๋ฃŒํ•œ ํ›„ ์ค‘๋ณต ํ–‰์ด ์กด์žฌํ•˜์ง€ ์•Š๋„๋ก ํ•˜๋ ค๋ฉด ๋‹ค์Œ ์ˆ˜๋™ ํ”„๋กœ์„ธ์Šค๋ฅผ ์‚ฌ์šฉํ•˜์„ธ์š”.

  1. insertId๋ฅผ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ์— ์—ด๋กœ ์ถ”๊ฐ€ํ•˜๊ณ  insertId ๊ฐ’์„ ๊ฐ ํ–‰์˜ ๋ฐ์ดํ„ฐ์— ํฌํ•จํ•ฉ๋‹ˆ๋‹ค.
  2. ์ŠคํŠธ๋ฆฌ๋ฐ์ด ์ค‘์ง€๋œ ํ›„ ๋‹ค์Œ ์ฟผ๋ฆฌ๋ฅผ ์ˆ˜ํ–‰ํ•˜์—ฌ ์ค‘๋ณต์„ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

    #standardSQL
    SELECT
      MAX(count) FROM(
      SELECT
        ID_COLUMN,
        count(*) as count
      FROM
        `TABLE_NAME`
      GROUP BY
        ID_COLUMN)

    ๊ฒฐ๊ณผ๊ฐ€ 1๋ณด๋‹ค ํฌ๋ฉด ์ค‘๋ณต์ด ์กด์žฌํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
  3. ์ค‘๋ณต์„ ์ œ๊ฑฐํ•˜๋ ค๋ฉด ๋‹ค์Œ ์ฟผ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค. ๋Œ€์ƒ ํ…Œ์ด๋ธ”์„ ์ง€์ •ํ•˜๊ณ  ๋Œ€๋Ÿ‰ ๊ฒฐ๊ณผ๋ฅผ ํ—ˆ์šฉํ•˜๊ณ  ๊ฒฐ๊ณผ ํ‰๋ฉดํ™”๋ฅผ ์‚ฌ์šฉ ์ค‘์ง€ํ•ฉ๋‹ˆ๋‹ค.

    #standardSQL
    SELECT
      * EXCEPT(row_number)
    FROM (
      SELECT
        *,
        ROW_NUMBER()
              OVER (PARTITION BY ID_COLUMN) row_number
      FROM
        `TABLE_NAME`)
    WHERE
      row_number = 1

์ค‘๋ณต ์‚ญ์ œ ์ฟผ๋ฆฌ ์ฐธ๊ณ ์‚ฌํ•ญ:

  • ์ค‘๋ณต ์ œ๊ฑฐ ์ฟผ๋ฆฌ๋ฅผ ์œ„ํ•œ ๋” ์•ˆ์ „ํ•œ ์ „๋žต์€ ์ƒˆ ํ…Œ์ด๋ธ”์„ ํƒ€๊ฒŸํŒ…ํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋˜๋Š” ์“ฐ๊ธฐ ์ฒ˜๋ฆฌ WRITE_TRUNCATE๋กœ ์†Œ์Šค ํ…Œ์ด๋ธ”์„ ํƒ€๊ฒŸํŒ…ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • ์ค‘๋ณต ์‚ญ์ œ ์ฟผ๋ฆฌ๋Š” ๊ฐ’์ด 1์ธ row_number ์—ด์„ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ ๋์— ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค. ์ฟผ๋ฆฌ๋Š” GoogleSQL์˜ SELECT * EXCEPT ๋ฌธ์„ ์‚ฌ์šฉํ•˜์—ฌ ๋Œ€์ƒ ํ…Œ์ด๋ธ”์—์„œ row_number ์—ด์„ ์ œ์™ธํ•ฉ๋‹ˆ๋‹ค. #standardSQL ํ”„๋ฆฌํ”ฝ์Šค๋Š” ์ด ์ฟผ๋ฆฌ์— GoogleSQL์„ ์‚ฌ์šฉ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ๋˜๋Š” ํŠน์ • ์—ด ์ด๋ฆ„๋ณ„๋กœ ์„ ํƒํ•˜์—ฌ ์ด ์—ด์„ ์ƒ๋žตํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • ์ค‘๋ณต์ด ์ œ๊ฑฐ๋œ ๋ผ์ด๋ธŒ ๋ฐ์ดํ„ฐ๋ฅผ ์ฟผ๋ฆฌํ•˜๋Š” ๊ฒฝ์šฐ ์ค‘๋ณต ์ œ๊ฑฐ ์ฟผ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํ…Œ์ด๋ธ”์—์„œ ๋ทฐ๋ฅผ ๋งŒ๋“ค ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ทฐ์— ๋Œ€ํ•œ ์ฟผ๋ฆฌ ๋น„์šฉ์€ ๋ทฐ์— ์„ ํƒ๋œ ์—ด์„ ๊ธฐ์ค€์œผ๋กœ ๊ณ„์‚ฐ๋˜๋ฉฐ ์ด๋กœ ์ธํ•ด ๊ฒ€์ƒ‰๋œ ๋ฐ”์ดํŠธ ํฌ๊ธฐ๊ฐ€ ์ปค์งˆ ์ˆ˜ ์žˆ์Œ์„ ์œ ์˜ํ•˜์„ธ์š”.

์‹œ๊ฐ„์œผ๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ์ŠคํŠธ๋ฆฌ๋ฐ

์‹œ๊ฐ„์œผ๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋ฉด ๊ฐ ํŒŒํ‹ฐ์…˜์— ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค. writeDisposition ์†์„ฑ์„ WRITE_TRUNCATE๋กœ ์„ค์ •ํ•˜์—ฌ ํŒŒํ‹ฐ์…˜์„ ๋ฎ์–ด์“ฐ๋Š” ๋กœ๋“œ, ์ฟผ๋ฆฌ ๋˜๋Š” ๋ณต์‚ฌ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•˜๋ฉด ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๊ฐ€ ์œ ์ง€๋ฉ๋‹ˆ๋‹ค. ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๋ฅผ ์‚ญ์ œํ•˜๋ ค๋ฉด ํŒŒํ‹ฐ์…˜์—์„œ tables.get์„ ํ˜ธ์ถœํ•˜์—ฌ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๊ฐ€ ๋น„์–ด ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๋‹ค.

์ˆ˜์ง‘ ์‹œ๊ฐ„์œผ๋กœ ํŒŒํ‹ฐ์…˜ ๋‚˜๋ˆ„๊ธฐ

์ˆ˜์ง‘ ์‹œ๊ฐ„์œผ๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ์ŠคํŠธ๋ฆฌ๋ฐํ•  ๋•Œ BigQuery๋Š” ํ˜„์žฌ UTC ์‹œ๊ฐ„์—์„œ ๋Œ€์ƒ ํŒŒํ‹ฐ์…˜์„ ์ถ”๋ก ํ•ฉ๋‹ˆ๋‹ค.

์ƒˆ๋กœ ๋„์ฐฉํ•˜๋Š” ๋ฐ์ดํ„ฐ๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ์— ์žˆ๋Š” ๋™์•ˆ ์ž„์‹œ๋กœ __UNPARTITIONED__ ํŒŒํ‹ฐ์…˜์— ๋ฐฐ์น˜๋ฉ๋‹ˆ๋‹ค. ํŒŒํ‹ฐ์…˜์œผ๋กœ ๋‚˜๋ˆ„์ง€ ์•Š์€ ๋ฐ์ดํ„ฐ๊ฐ€ ์ถฉ๋ถ„ํ•œ ๊ฒฝ์šฐ BigQuery๋Š” ๋ฐ์ดํ„ฐ๋ฅผ ํŒŒํ‹ฐ์…˜ ๋‚˜๋ˆ„๊ธฐํ•˜์—ฌ ์˜ฌ๋ฐ”๋ฅธ ํŒŒํ‹ฐ์…˜์— ๋ฐฐ์น˜ํ•ฉ๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ __UNPARTITIONED__ ํŒŒํ‹ฐ์…˜์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ด๋™ํ•˜๋Š” ๋ฐ ๊ฑธ๋ฆฌ๋Š” ์‹œ๊ฐ„์— ๋Œ€ํ•œ SLA๋Š” ์—†์Šต๋‹ˆ๋‹ค. ์ฟผ๋ฆฌ๋Š” ์œ ์‚ฌ ์—ด(์„ ํ˜ธํ•˜๋Š” ๋ฐ์ดํ„ฐ ์œ ํ˜•์— ๋”ฐ๋ผ _PARTITIONTIME ๋˜๋Š” _PARTITIONDATE) ์ค‘ ํ•˜๋‚˜๋ฅผ ์‚ฌ์šฉํ•ด NULL ๊ฐ’์„ __UNPARTITIONED__ ํŒŒํ‹ฐ์…˜์—์„œ ํ•„ํ„ฐ๋งํ•จ์œผ๋กœ์จ ์ฟผ๋ฆฌ์—์„œ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ์˜ ๋ฐ์ดํ„ฐ๋ฅผ ์ œ์™ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ผ๋ณ„๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๊ฒฝ์šฐ ํŒŒํ‹ฐ์…˜ ๋ฐ์ฝ”๋ ˆ์ดํ„ฐ๋ฅผ insertAll ์š”์ฒญ์— ํฌํ•จ์‹œ์ผœ ์ถ”๊ฐ€ํ•˜๋ฉด ๋‚ ์งœ ์ถ”๋ก ์„ ์žฌ์ •์˜ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. tableId ๋งค๊ฐœ๋ณ€์ˆ˜์— ๋ฐ์ฝ”๋ ˆ์ดํ„ฐ๋ฅผ ํฌํ•จํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด table1 ํ…Œ์ด๋ธ”์—์„œ ํŒŒํ‹ฐ์…˜ ๋ฐ์ฝ”๋ ˆ์ดํ„ฐ๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด 2021-03-01์— ํ•ด๋‹นํ•˜๋Š” ํŒŒํ‹ฐ์…˜์— ์ŠคํŠธ๋ฆฌ๋ฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

table1$20210301

ํŒŒํ‹ฐ์…˜ ๋ฐ์ฝ”๋ ˆ์ดํ„ฐ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๊ฒฝ์šฐ ํ˜„์žฌ UTC ์‹œ๊ฐ„์„ ๊ธฐ์ค€์œผ๋กœ ์ง€๋‚œ 31์ผ ๋ฐ ํ–ฅํ›„ 16์ผ ์ด๋‚ด์˜ ํŒŒํ‹ฐ์…˜์œผ๋กœ ์ŠคํŠธ๋ฆฌ๋ฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ํ—ˆ์šฉ ๋ฒ”์œ„ ์ด์™ธ์˜ ๋‚ ์งœ์— ํŒŒํ‹ฐ์…˜์— ๊ธฐ๋กํ•˜๋ ค๋ฉด ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ” ๋ฐ์ดํ„ฐ์— ์ถ”๊ฐ€ ๋ฐ ๋ฎ์–ด์“ฐ๊ธฐ์— ์„ค๋ช…๋œ ๋Œ€๋กœ ๋กœ๋“œ ๋˜๋Š” ์ฟผ๋ฆฌ ์ž‘์—…์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

ํŒŒํ‹ฐ์…˜ ๋ฐ์ฝ”๋ ˆ์ดํ„ฐ๋ฅผ ์‚ฌ์šฉํ•œ ์ŠคํŠธ๋ฆฌ๋ฐ์€ ๋งค์ผ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”์—๋งŒ ์ง€์›๋ฉ๋‹ˆ๋‹ค. ์‹œ๊ฐ„๋ณ„, ์›”๊ฐ„ ๋˜๋Š” ์—ฐ๊ฐ„ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”์—์„œ๋Š” ์ง€์›๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

ํ…Œ์ŠคํŠธ์—๋Š” bq ๋ช…๋ น์ค„ ๋„๊ตฌ bq insert CLI ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ๋‹ค์Œ ๋ช…๋ น์–ด๋Š” 2017๋…„ 1์›” 1์ผ($20170101) ๋‚ ์งœ์˜ ํŒŒํ‹ฐ์…˜์— ๋Œ€ํ•œ ๋‹จ์ผ ํ–‰์„ mydataset.mytable์ด๋ผ๋Š” ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ์ŠคํŠธ๋ฆฌ๋ฐํ•ฉ๋‹ˆ๋‹ค.

echo '{"a":1, "b":2}' | bq insert 'mydataset.mytable$20170101'

์‹œ๊ฐ„ ๋‹จ์œ„ ์—ด๋กœ ํŒŒํ‹ฐ์…˜ ๋‚˜๋ˆ„๊ธฐ

์ง€๋‚œ 10๋…„๋ถ€ํ„ฐ ํ–ฅํ›„ 1๋…„ ์ด๋‚ด์˜ DATE, DATETIME, TIMESTAMP ์—ด์„ ๊ธฐ์ค€์œผ๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ๋ฒ”์œ„๋ฅผ ๋ฒ—์–ด๋‚˜๋Š” ๋ฐ์ดํ„ฐ๋Š” ๊ฑฐ๋ถ€๋ฉ๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ๊ฐ€ ์ŠคํŠธ๋ฆฌ๋ฐ๋˜๋ฉด ์ฒ˜์Œ์—๋Š” __UNPARTITIONED__ ํŒŒํ‹ฐ์…˜์— ๋ฐฐ์น˜๋ฉ๋‹ˆ๋‹ค. ํŒŒํ‹ฐ์…˜์œผ๋กœ ๋‚˜๋ˆ„์ง€ ์•Š์€ ๋ฐ์ดํ„ฐ๊ฐ€ ์ถฉ๋ถ„ํ•œ ๊ฒฝ์šฐ BigQuery๋Š” ์ž๋™์œผ๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ๋‹ค์‹œ ํŒŒํ‹ฐ์…˜ ๋‚˜๋ˆ„๊ธฐํ•˜์—ฌ ์ ์ ˆํ•œ ํŒŒํ‹ฐ์…˜์— ๋ฐฐ์น˜ํ•ฉ๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ __UNPARTITIONED__ ํŒŒํ‹ฐ์…˜์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ด๋™ํ•˜๋Š” ๋ฐ ๊ฑธ๋ฆฌ๋Š” ์‹œ๊ฐ„์— ๋Œ€ํ•œ SLA๋Š” ์—†์Šต๋‹ˆ๋‹ค.

  • ์ฐธ๊ณ : ์ผ๋ณ„ ํŒŒํ‹ฐ์…˜์€ ์‹œ๊ฐ„๋ณ„, ์›”๊ฐ„, ์—ฐ๊ฐ„ ํŒŒํ‹ฐ์…˜๊ณผ ๋‹ค๋ฅด๊ฒŒ ์ฒ˜๋ฆฌ๋ฉ๋‹ˆ๋‹ค. ๊ธฐ๊ฐ„(์ง€๋‚œ 7์ผ์—์„œ ํ–ฅํ›„ 3์ผ)์„ ๋ฒ—์–ด๋‚˜๋Š” ๋ฐ์ดํ„ฐ๋งŒ UNPARTITIONED ํŒŒํ‹ฐ์…˜์œผ๋กœ ์ถ”์ถœ๋˜๊ณ  ๋‹ค์‹œ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆ„๊ธฐ๋ฅผ ๋Œ€๊ธฐํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜๋ฉด ์‹œ๊ฐ„๋ณ„๋กœ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ”์˜ ๊ฒฝ์šฐ ๋ฐ์ดํ„ฐ๊ฐ€ ํ•ญ์ƒ UNPARTITIONED ํŒŒํ‹ฐ์…˜์œผ๋กœ ์ถ”์ถœ๋˜๊ณ  ๋‚˜์ค‘์— ๋‹ค์‹œ ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆ•๋‹ˆ๋‹ค.

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜์—ฌ ์ž๋™์œผ๋กœ ํ…Œ์ด๋ธ” ๋งŒ๋“ค๊ธฐ

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์€ ๋…ผ๋ฆฌ์  ํ…Œ์ด๋ธ”์„ ๋” ์ž‘์€ ์—ฌ๋Ÿฌ ํ…Œ์ด๋ธ”๋กœ ๋ถ„ํ• ํ•˜์—ฌ ๋” ์ž‘์€ ๋ฐ์ดํ„ฐ ์ง‘ํ•ฉ์„ ๋งŒ๋“œ๋Š” ๋ฉ”์ปค๋‹ˆ์ฆ˜์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค(์˜ˆ: ์‚ฌ์šฉ์ž ID). ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์—๋Š” ์•„๋ž˜ ์„ค๋ช…๋œ ์—ฌ๋Ÿฌ ์ œํ•œ ์‚ฌํ•ญ์ด ํฌํ•จ๋ฉ๋‹ˆ๋‹ค. ๋Œ€์‹  ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ” ๋ฐ ํด๋Ÿฌ์Šคํ„ฐ๋ง๋œ ํ…Œ์ด๋ธ”์ด ๋™์ž‘์„ ์ˆ˜ํ–‰ํ•˜๊ธฐ ์œ„ํ•ด ๊ถŒ์žฅ๋˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค.

BigQuery API๋ฅผ ํ†ตํ•ด ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜๋ ค๋ฉด insertAll ์š”์ฒญ์— templateSuffix ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค. bq ๋ช…๋ น์ค„ ๋„๊ตฌ์˜ ๊ฒฝ์šฐ insert ๋ช…๋ น์–ด์— template_suffix ํ”Œ๋ž˜๊ทธ๋ฅผ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค. BigQuery๋Š” templateSuffix ๋งค๊ฐœ๋ณ€์ˆ˜ ๋˜๋Š” template_suffix ํ”Œ๋ž˜๊ทธ๋ฅผ ๊ฐ์ง€ํ•˜๋ฉด ๋Œ€์ƒ ํ…Œ์ด๋ธ”์„ ๊ธฐ๋ณธ ํ…œํ”Œ๋ฆฟ์œผ๋กœ ์ทจ๊ธ‰ํ•ฉ๋‹ˆ๋‹ค. ๋Œ€์ƒ ํ…Œ์ด๋ธ”๊ณผ ๋™์ผํ•œ ์Šคํ‚ค๋งˆ๋ฅผ ๊ณต์œ ํ•˜๋ฉฐ ์ง€์ •๋œ ์„œํ”ฝ์Šค๋ฅผ ํฌํ•จํ•˜๋Š” ์ด๋ฆ„์ด ์žˆ๋Š” ์ƒˆ ํ…Œ์ด๋ธ”์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค.

<targeted_table_name> + <templateSuffix>

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜๋ฉด ๊ฐ ํ…Œ์ด๋ธ”์„ ๊ฐœ๋ณ„์ ์œผ๋กœ ๋งŒ๋“ค๊ณ  ๊ฐ ํ…Œ์ด๋ธ”์˜ ์Šคํ‚ค๋งˆ๋ฅผ ์ง€์ •ํ•˜๋Š” ์˜ค๋ฒ„ํ—ค๋“œ๊ฐ€ ๋ฐฉ์ง€๋ฉ๋‹ˆ๋‹ค. ํ•˜๋‚˜์˜ ํ…œํ”Œ๋ฆฟ๋งŒ ๋งŒ๋“ค๊ณ  ๋‹ค์–‘ํ•œ ์„œํ”ฝ์Šค๋ฅผ ์ œ๊ณตํ•˜์—ฌ BigQuery์—์„œ ์ƒˆ ํ…Œ์ด๋ธ”์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ๋„๋ก ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค. BigQuery๋Š” ๋™์ผํ•œ ํ”„๋กœ์ ํŠธ ๋ฐ ๋ฐ์ดํ„ฐ์„ธํŠธ์— ํ…Œ์ด๋ธ”์„ ๋ฐฐ์น˜ํ•ฉ๋‹ˆ๋‹ค.

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜์—ฌ ๋งŒ๋“ค์–ด์ง„ ํ…Œ์ด๋ธ”์€ ์ผ๋ฐ˜์ ์œผ๋กœ ๋ช‡ ์ดˆ ๋‚ด์— ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋“œ๋ฌผ์ง€๋งŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ฒŒ ๋˜๊ธฐ๊นŒ์ง€ ์‹œ๊ฐ„์ด ๋” ๊ฑธ๋ฆฌ๋Š” ๊ฒฝ์šฐ๋„ ์žˆ์Šต๋‹ˆ๋‹ค.

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ ๋ณ€๊ฒฝ

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ๋ฅผ ๋ณ€๊ฒฝํ•˜๋ฉด ์ดํ›„์— ์ƒ์„ฑ๋˜๋Š” ๋ชจ๋“  ํ…Œ์ด๋ธ”์€ ์—…๋ฐ์ดํŠธ๋œ ์Šคํ‚ค๋งˆ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ์กด ํ…Œ์ด๋ธ”์— ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๊ฐ€ ์—ฌ์ „ํžˆ ์žˆ๋Š” ๊ฒฝ์šฐ๋ฅผ ์ œ์™ธํ•˜๊ณ  ์ด์ „์— ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์€ ์˜ํ–ฅ์„ ๋ฐ›์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

์•„์ง ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ๊ฐ€ ์žˆ๋Š” ๊ธฐ์กด ํ…Œ์ด๋ธ”์˜ ๊ฒฝ์šฐ ํ•˜์œ„ ํ˜ธํ™˜ ๊ฐ€๋Šฅํ•œ ๋ฐฉ์‹์œผ๋กœ ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ๋ฅผ ์ˆ˜์ •ํ•˜๋ฉด ์ด๋Ÿฌํ•œ ํ™œ์„ฑ ์ŠคํŠธ๋ฆฌ๋ฐ ์ƒ์„ฑ ํ…Œ์ด๋ธ”์˜ ์Šคํ‚ค๋งˆ๋„ ์—…๋ฐ์ดํŠธ๋ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์—ญํ˜ธํ™˜๋˜์ง€ ์•Š๋Š” ๋ฐฉ์‹์œผ๋กœ ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ๋ฅผ ์ˆ˜์ •ํ•˜๋Š” ๊ฒฝ์šฐ ๊ธฐ์กด ์Šคํ‚ค๋งˆ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๋ฒ„ํผ๋ง๋œ ๋ฐ์ดํ„ฐ๋Š” ์†์‹ค๋ฉ๋‹ˆ๋‹ค. ๋˜ํ•œ ํ˜„์žฌ ํ˜ธํ™˜๋˜์ง€ ์•Š๋Š” ์ด์ „ ์Šคํ‚ค๋งˆ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ธฐ์กด์— ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”๋กœ ์ƒˆ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์Šคํ‚ค๋งˆ๋ฅผ ๋ณ€๊ฒฝํ•œ ํ›„ ์ฟผ๋ฆฌ ์ƒ์„ฑ ํ…Œ์ด๋ธ”์— ์ƒˆ ๋ฐ์ดํ„ฐ๋ฅผ ์‚ฝ์ž…ํ•˜๊ธฐ ์ „์— ๋ณ€๊ฒฝ ์‚ฌํ•ญ์ด ์ „ํŒŒ๋  ๋•Œ๊นŒ์ง€ ๊ธฐ๋‹ค๋ฆฌ์„ธ์š”. ์ƒˆ ํ•„๋“œ ์‚ฝ์ž… ์š”์ฒญ์€ ๋ช‡ ๋ถ„ ์ด๋‚ด์— ์„ฑ๊ณต์ ์œผ๋กœ ์ˆ˜ํ–‰๋ฉ๋‹ˆ๋‹ค. ์ƒˆ ํ•„๋“œ๋ฅผ ์ฟผ๋ฆฌํ•˜๋ ค๋ฉด ์ตœ๋Œ€ 90๋ถ„๊นŒ์ง€ ๊ธฐ๋‹ค๋ ค์•ผ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์˜ ์Šคํ‚ค๋งˆ๋ฅผ ๋ณ€๊ฒฝํ•˜๋ ค๋Š” ๊ฒฝ์šฐ, ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ํ†ตํ•œ ์ŠคํŠธ๋ฆฌ๋ฐ์ด ์ค‘๋‹จ๋˜๊ณ  ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์˜ ์ŠคํŠธ๋ฆฌ๋ฐ ํ†ต๊ณ„ ์„น์…˜์ด tables.get() ์‘๋‹ต์— ์—†๋Š” ๊ฒฝ์šฐ(ํ…Œ์ด๋ธ”์— ๋ฒ„ํผ๋ง๋œ ๋ฐ์ดํ„ฐ๊ฐ€ ์—†์Œ์„ ์˜๋ฏธ) ์™ธ์—๋Š” ์Šคํ‚ค๋งˆ๋ฅผ ๋ณ€๊ฒฝํ•˜์ง€ ๋งˆ์„ธ์š”.

ํŒŒํ‹ฐ์…˜์„ ๋‚˜๋ˆˆ ํ…Œ์ด๋ธ” ๋ฐ ํด๋Ÿฌ์Šคํ„ฐ๋ง๋œ ํ…Œ์ด๋ธ”์€ ์•ž์˜ ์ œํ•œ ์‚ฌํ•ญ์ด ์ ์šฉ๋˜์ง€ ์•Š์œผ๋ฉฐ, ๊ถŒ์žฅ๋˜๋Š” ๋ฉ”์ปค๋‹ˆ์ฆ˜์ž…๋‹ˆ๋‹ค.

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ” ์„ธ๋ถ€์ •๋ณด

ํ…œํ”Œ๋ฆฟ ์„œํ”ฝ์Šค ๊ฐ’
templateSuffix(๋˜๋Š” --template_suffix) ๊ฐ’์€ ๋ฌธ์ž(a-z, A-Z), ์ˆซ์ž(0-9), ๋ฐ‘์ค„(_)๋งŒ ํฌํ•จํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ํ…Œ์ด๋ธ” ์ด๋ฆ„๊ณผ ํ…Œ์ด๋ธ” ์„œํ”ฝ์Šค๋ฅผ ๊ฒฐํ•ฉํ•œ ์ตœ๋Œ€ ๊ธธ์ด๋Š” 1,024์ž์ž…๋‹ˆ๋‹ค.
ํ• ๋‹น๋Ÿ‰

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์—๋Š” ์ŠคํŠธ๋ฆฌ๋ฐ ํ• ๋‹น๋Ÿ‰ ์ œํ•œ์ด ์ ์šฉ๋ฉ๋‹ˆ๋‹ค. ํ”„๋กœ์ ํŠธ๋Š” tables.insert API์™€ ๋น„์Šทํ•˜๊ฒŒ ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜์—ฌ ์ดˆ๋‹น ์ตœ๋Œ€ 10๊ฐœ์˜ ํ…Œ์ด๋ธ”์„ ๋งŒ๋“ค ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ํ• ๋‹น๋Ÿ‰์€ ์ˆ˜์ •๋˜๋Š” ํ…Œ์ด๋ธ”์ด ์•„๋‹Œ ์ƒ์„ฑ๋˜๋Š” ํ…Œ์ด๋ธ”์—๋งŒ ์ ์šฉ๋ฉ๋‹ˆ๋‹ค.

์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์ด ํ…Œ์ด๋ธ”์„ ์ดˆ๋‹น 10๊ฐœ ๋„˜๊ฒŒ ๋งŒ๋“ค์–ด์•ผ ํ•  ๊ฒฝ์šฐ ํด๋Ÿฌ์Šคํ„ฐ๋ง๋œ ํ…Œ์ด๋ธ”์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์ด ์ข‹์Šต๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ๋‹จ์ผ ํด๋Ÿฌ์Šคํ„ฐ๋ง ํ…Œ์ด๋ธ”์˜ ํ‚ค ์—ด์— ๋†’์€ ์นด๋””๋„๋ฆฌํ‹ฐ ํ…Œ์ด๋ธ” ID๋ฅผ ๋ฐฐ์น˜ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋ผ์ด๋ธŒ๊นŒ์ง€์˜ ์‹œ๊ฐ„

์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์€ ๋ฐ์ดํ„ฐ ์„ธํŠธ์—์„œ ๋งŒ๋ฃŒ ์‹œ๊ฐ„์„ ์ƒ์†ํ•ฉ๋‹ˆ๋‹ค. ์ผ๋ฐ˜์ ์ธ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ์™€ ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์„ ์ฆ‰์‹œ ๋ณต์‚ฌํ•  ์ˆ˜๋Š” ์—†์Šต๋‹ˆ๋‹ค.

์ค‘๋ณต ์‚ญ์ œ

์ค‘๋ณต ์‚ญ์ œ๋Š” ๋Œ€์ƒ ํ…Œ์ด๋ธ”์— ๋Œ€ํ•œ ๋™์ผํ•œ ์ฐธ์กฐ ๊ฐ„์—๋งŒ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”๊ณผ ์ •๊ทœ insertAll ๋ช…๋ น์–ด๋ฅผ ๋ชจ๋‘ ์‚ฌ์šฉํ•˜์—ฌ ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”๋กœ ๋™์‹œ์— ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋Š” ๊ฒฝ์šฐ ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”๊ณผ ์ •๊ทœ insertAll ๋ช…๋ น์–ด๋กœ ์‚ฝ์ž…๋œ ํ–‰ ๊ฐ„์—๋Š” ์ค‘๋ณต ์‚ญ์ œ๊ฐ€ ๋ฐœ์ƒํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

๋ทฐ

ํ…œํ”Œ๋ฆฟ ํ…Œ์ด๋ธ”๊ณผ ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์€ ๋ทฐ์ผ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.

์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž… ๋ฌธ์ œ ํ•ด๊ฒฐ

๋‹ค์Œ ์„น์…˜์—์„œ๋Š” ๊ธฐ์กด ์ŠคํŠธ๋ฆฌ๋ฐ API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ BigQuery๋กœ ๋ฐ์ดํ„ฐ๋ฅผ ์ŠคํŠธ๋ฆฌ๋ฐํ•  ๋•Œ ๋ฐœ์ƒํ•˜๋Š” ์˜ค๋ฅ˜๋ฅผ ํ•ด๊ฒฐํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค. ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์˜ ํ• ๋‹น๋Ÿ‰ ์˜ค๋ฅ˜๋ฅผ ํ•ด๊ฒฐํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž… ํ• ๋‹น๋Ÿ‰ ์˜ค๋ฅ˜๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

์‹คํŒจ HTTP ์‘๋‹ต ์ฝ”๋“œ

๋„คํŠธ์›Œํฌ ์˜ค๋ฅ˜์™€ ๊ฐ™์€ ์‹คํŒจ HTTP ์‘๋‹ต ์ฝ”๋“œ๋ฅผ ๋ฐ›๋Š” ๊ฒฝ์šฐ ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์ด ์„ฑ๊ณต์ ์œผ๋กœ ์ˆ˜ํ–‰๋˜์—ˆ๋Š”์ง€ ์—ฌ๋ถ€๋ฅผ ํ™•์ธํ•  ๋ฐฉ๋ฒ•์ด ์—†์Šต๋‹ˆ๋‹ค. ์š”์ฒญ์„ ๋‹ค์‹œ ์ „์†กํ•˜๋ ค๊ณ  ์‹œ๋„ํ•˜๋Š” ๊ฒฝ์šฐ ํ…Œ์ด๋ธ”์— ์ค‘๋ณต๋œ ํ–‰์ด ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํ…Œ์ด๋ธ”์—์„œ ์ค‘๋ณต์ด ๋ฐœ์ƒํ•˜์ง€ ์•Š๋„๋ก ํ•˜๋ ค๋ฉด ์š”์ฒญ์„ ๋ณด๋‚ผ ๋•Œ insertId ์†์„ฑ์„ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. BigQuery๋Š” ์ค‘๋ณต ์‚ญ์ œ์— insertId ์†์„ฑ์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

๊ถŒํ•œ ์˜ค๋ฅ˜, ์ž˜๋ชป๋œ ํ…Œ์ด๋ธ” ์ด๋ฆ„ ์˜ค๋ฅ˜ ๋˜๋Š” ํ• ๋‹น๋Ÿ‰ ์ดˆ๊ณผ ์˜ค๋ฅ˜๋ฅผ ๋ฐ›๋Š” ๊ฒฝ์šฐ ํ–‰์ด ์‚ฝ์ž…๋˜์ง€ ์•Š์œผ๋ฉฐ ์ „์ฒด ์š”์ฒญ์ด ์‹คํŒจํ•ฉ๋‹ˆ๋‹ค.

์„ฑ๊ณต HTTP ์‘๋‹ต ์ฝ”๋“œ

BigQuery์—์„œ ๋ถ€๋ถ„์ ์œผ๋กœ๋งŒ ํ–‰ ์‚ฝ์ž…์— ์„ฑ๊ณตํ•˜๋Š” ๊ฒฝ์šฐ๋„ ์žˆ์œผ๋ฏ€๋กœ ์„ฑ๊ณต HTTP ์‘๋‹ต ์ฝ”๋“œ๋ฅผ ๋ฐ›๋”๋ผ๋„ ์‘๋‹ต์˜ insertErrors ์†์„ฑ์„ ํ™•์ธํ•˜์—ฌ ํ–‰ ์‚ฝ์ž…์ด ์„ฑ๊ณต์ ์œผ๋กœ ์ˆ˜ํ–‰๋˜์—ˆ๋Š”์ง€ ์—ฌ๋ถ€๋ฅผ ํŒ๋‹จํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋‹ค์Œ ์‹œ๋‚˜๋ฆฌ์˜ค ์ค‘ ํ•˜๋‚˜๊ฐ€ ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

  • ๋ชจ๋“  ํ–‰์ด ์‚ฝ์ž…๋˜์—ˆ์Šต๋‹ˆ๋‹ค. insertErrors ์†์„ฑ์ด ๋นˆ ๋ชฉ๋ก์ธ ๊ฒฝ์šฐ ๋ชจ๋“  ํ–‰์ด ์„ฑ๊ณต์ ์œผ๋กœ ์‚ฝ์ž…๋œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.
  • ์ผ๋ถ€ ํ–‰์ด ์‚ฝ์ž…๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ํ–‰์— ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜๊ฐ€ ์žˆ๋Š” ๊ฒฝ์šฐ๋ฅผ ์ œ์™ธํ•˜๊ณ  insertErrors ์†์„ฑ์— ํ‘œ์‹œ๋œ ํ–‰์€ ์‚ฝ์ž…๋˜์ง€ ์•Š์•˜๊ณ  ๋‹ค๋ฅธ ๋ชจ๋“  ํ–‰์€ ์„ฑ๊ณต์ ์œผ๋กœ ์‚ฝ์ž…๋˜์—ˆ์Šต๋‹ˆ๋‹ค. errors ์†์„ฑ์—๋Š” ์„ฑ๊ณตํ•˜์ง€ ๋ชปํ•œ ํ–‰์˜ ์‹คํŒจ ์ด์œ ์— ๋Œ€ํ•œ ์ž์„ธํ•œ ์ •๋ณด๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. index ์†์„ฑ์€ ์˜ค๋ฅ˜๊ฐ€ ์ ์šฉ๋˜๋Š” ์š”์ฒญ์˜ 0 ๊ธฐ๋ฐ˜ ํ–‰ ์ƒ‰์ธ์„ ๋‚˜ํƒ€๋ƒ…๋‹ˆ๋‹ค.
  • ์„ฑ๊ณต์ ์œผ๋กœ ์‚ฝ์ž…๋œ ํ–‰์ด ์—†์Šต๋‹ˆ๋‹ค. BigQuery์—์„œ ์š”์ฒญ์˜ ๊ฐœ๋ณ„ ํ–‰์— ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜๊ฐ€ ๋ฐœ์ƒํ•˜๋Š” ๊ฒฝ์šฐ ์•„๋ฌด๋Ÿฐ ํ–‰์ด ์‚ฝ์ž…๋˜์ง€ ์•Š์œผ๋ฉฐ ํ–‰์— ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜๊ฐ€ ์—†๋”๋ผ๋„ ๊ฐ ํ–‰์— insertErrors ํ•ญ๋ชฉ์ด ๋ฐ˜ํ™˜๋ฉ๋‹ˆ๋‹ค. ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜๊ฐ€ ์—†๋Š” ํ–‰์˜ ๊ฒฝ์šฐ reason ์†์„ฑ์ด stopped๋กœ ์„ค์ •๋œ ์˜ค๋ฅ˜๊ฐ€ ์žˆ์œผ๋ฉฐ ํ˜„์žฌ ์ƒํƒœ ๊ทธ๋Œ€๋กœ ๋‹ค์‹œ ๋ณด๋‚ผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์‹คํŒจํ•œ ํ–‰์—๋Š” ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜์— ๋Œ€ํ•œ ์ž์„ธํ•œ ์ •๋ณด๊ฐ€ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค. ๊ฐ BigQuery ๋ฐ์ดํ„ฐ ์œ ํ˜•์— ์ง€์›๋˜๋Š” ํ”„๋กœํ† ์ฝœ ๋ฒ„ํผ ์œ ํ˜•์— ๋Œ€ํ•ด ์•Œ์•„๋ณด๋ ค๋ฉด ๋ฐ์ดํ„ฐ ์œ ํ˜• ๋ณ€ํ™˜์„ ์ฐธ์กฐํ•˜์„ธ์š”.

์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์˜ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ์˜ค๋ฅ˜

BigQuery์˜ ์ŠคํŠธ๋ฆฌ๋ฐ API๋Š” ๋†’์€ ์‚ฝ์ž… ๋น„์œจ์— ๋งž๊ฒŒ ๊ณ ์•ˆ๋˜์—ˆ์œผ๋ฏ€๋กœ ์ŠคํŠธ๋ฆฌ๋ฐ ์‹œ์Šคํ…œ๊ณผ ์ƒํ˜ธ์ž‘์šฉ ์‹œ ๊ธฐ๋ฐ˜ ํ…Œ์ด๋ธ” ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ๋…ธ์ถœ์˜ ์ˆ˜์ •์€ eventual consistency๋ฅผ ๊ฐ–๊ฒŒ ๋ฉ๋‹ˆ๋‹ค. ๋Œ€๋ถ€๋ถ„์˜ ๊ฒฝ์šฐ ๋ฉ”ํƒ€๋ฐ์ดํ„ฐ ๋ณ€๊ฒฝ์€ ๋ช‡ ๋ถ„ ์ด๋‚ด์— ์ „ํŒŒ๋˜์ง€๋งŒ ์ด ์‹œ๊ฐ„ ๋™์•ˆ API ์‘๋‹ต์€ ๋น„์ผ๊ด€์ ์ธ ํ…Œ์ด๋ธ” ์ƒํƒœ๋ฅผ ๋ฐ˜์˜ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ผ๋ถ€ ์‹œ๋‚˜๋ฆฌ์˜ค์—๋Š” ๋‹ค์Œ์ด ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.

  • ์Šคํ‚ค๋งˆ ๋ณ€๊ฒฝ์‚ฌํ•ญ. ์ŠคํŠธ๋ฆฌ๋ฐ ์‹œ์Šคํ…œ์€ ์Šคํ‚ค๋งˆ ๋ณ€๊ฒฝ์„ ์ฆ‰๊ฐ ์ ์šฉํ•˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ ์ตœ๊ทผ ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์„ ์ˆ˜์‹ ํ•œ ํ…Œ์ด๋ธ”์˜ ์Šคํ‚ค๋งˆ๋ฅผ ์ˆ˜์ •ํ•˜๋Š” ๊ฒฝ์šฐ ์Šคํ‚ค๋งˆ ๋ถˆ์ผ์น˜ ์˜ค๋ฅ˜ ์‘๋‹ต์ด ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • ํ…Œ์ด๋ธ” ๋งŒ๋“ค๊ธฐ/์‚ญ์ œ. ์กด์žฌํ•˜์ง€ ์•Š๋Š” ํ…Œ์ด๋ธ”๋กœ ์ŠคํŠธ๋ฆฌ๋ฐํ•˜๋ฉด notFound ์‘๋‹ต ๋ณ€ํ˜•์ด ๋ฐ˜ํ™˜๋ฉ๋‹ˆ๋‹ค. ์‘๋‹ต์œผ๋กœ ์ƒ์„ฑ๋œ ํ…Œ์ด๋ธ”์€ ํ›„์† ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์—์„œ ์ฆ‰์‹œ ์ธ์‹๋˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ ํ…Œ์ด๋ธ”์„ ์‚ญ์ œํ•˜๊ฑฐ๋‚˜ ๋‹ค์‹œ ๋งŒ๋“ค์–ด๋„ ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์ด ์‹ค์ œ๋กœ ์ด์ „ ํ…Œ์ด๋ธ”๋กœ ์ „๋‹ฌ๋˜๋Š” ๊ธฐ๊ฐ„์ด ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์ด ์ƒˆ ํ…Œ์ด๋ธ”์— ์กด์žฌํ•˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • ํ…Œ์ด๋ธ” ์ž˜๋ฆผ. ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ WRITE_TRUNCATE์˜ writeDisposition์„ ์‚ฌ์šฉํ•˜๋Š” ์ฟผ๋ฆฌ ์ž‘์—…์„ ์‚ฌ์šฉํ•ด์„œ ํ…Œ์ด๋ธ” ๋ฐ์ดํ„ฐ๋ฅผ ์ž๋ฅด๋ฉด ์ผ๊ด€์„ฑ ๊ธฐ๊ฐ„ ๋™์•ˆ ํ›„์† ์‚ฝ์ž…์ด ๋ˆ„๋ฝ๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ ๋ˆ„๋ฝ/์‚ฌ์šฉ ๋ถˆ๊ฐ€

์ŠคํŠธ๋ฆฌ๋ฐ ์‚ฝ์ž…์€ ์ผ์‹œ์ ์œผ๋กœ ์“ฐ๊ธฐ ์ตœ์ ํ™” ์Šคํ† ๋ฆฌ์ง€์— ์œ„์น˜ํ•˜๋Š”๋ฐ, ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฒ„ํผ์˜ ๊ฐ€์šฉ์„ฑ ํŠน์„ฑ์€ ๊ด€๋ฆฌ ์Šคํ† ๋ฆฌ์ง€์™€๋Š” ๋‹ค๋ฆ…๋‹ˆ๋‹ค. ํ…Œ์ด๋ธ” ๋ณต์‚ฌ ์ž‘์—… ๋ฐ tabledata.list์™€ ๊ฐ™์€ API ๋ฉ”์„œ๋“œ ๋“ฑ BigQuery์˜ ํŠน์ • ์ž‘์—…์€ ์“ฐ๊ธฐ ์ตœ์ ํ™” ์Šคํ† ๋ฆฌ์ง€์™€ ์ƒํ˜ธ์ž‘์šฉํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ์ตœ๊ทผ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ๊ฐ€ ๋Œ€์ƒ ํ…Œ์ด๋ธ” ๋˜๋Š” ์ถœ๋ ฅ์— ์กด์žฌํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.