์‹คํ—˜ ์‹คํ–‰์— ์ˆ˜๋™์œผ๋กœ ๋ฐ์ดํ„ฐ ๋กœ๊น…

๋กœ๊น…์„ ์œ„ํ•ด Python์šฉ Vertex AI SDK๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

์ง€์›๋˜๋Š” ์ธก์ •ํ•ญ๋ชฉ ๋ฐ ๋งค๊ฐœ๋ณ€์ˆ˜:

  • ์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ
  • ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ
  • ๋งค๊ฐœ๋ณ€์ˆ˜
  • ๋ถ„๋ฅ˜ ์ธก์ •ํ•ญ๋ชฉ

Python์šฉ Vertex AI SDK

์ฐธ๊ณ : ์„ ํƒ์  resume ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ TRUE๋กœ ์ง€์ •ํ•˜๋ฉด ์ด์ „์— ์‹œ์ž‘๋œ ์‹คํ–‰์ด ๋‹ค์‹œ ์‹œ์ž‘๋ฉ๋‹ˆ๋‹ค. ์ง€์ •๋˜์ง€ ์•Š์€ ๊ฒฝ์šฐ resume์˜ ๊ธฐ๋ณธ๊ฐ’์€ FALSE์ด๊ณ  ์ƒˆ ์‹คํ–‰์ด ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.

๋‹ค์Œ ์ƒ˜ํ”Œ์€ aiplatform ํ•จ์ˆ˜์—์„œ init ๋ฉ”์„œ๋“œ๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ

์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ์€ ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ ์˜†์— ์ €์žฅ๋˜๋Š” ๋‹จ์ผ ๊ฐ’ ์Šค์นผ๋ผ ์ธก์ •ํ•ญ๋ชฉ์ด๋ฉฐ ์‹คํ—˜ ์‹คํ–‰์˜ ์ตœ์ข… ์š”์•ฝ์„ ๋‚˜ํƒ€๋ƒ…๋‹ˆ๋‹ค.

ํ•œ ๊ฐ€์ง€ ์‚ฌ์šฉ ์‚ฌ๋ก€๋Š” ์ธ๋‚ด ๊ตฌ์„ฑ์ด ์ง€์†์ ์ธ ํ•™์Šต์„ ํ—ˆ์šฉํ•˜์ง€๋งŒ ์ตœ์‹  ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ์ด ๋ณต์›๋œ ๋ชจ๋ธ์„ ๋Œ€ํ‘œํ•˜์ง€ ์•Š๊ธฐ ๋•Œ๋ฌธ์— ํ›„๋ณด ๋ชจ๋ธ์ด ์ดˆ๊ธฐ ๋‹จ๊ณ„์—์„œ ๋ณต์›๋˜๊ณ  ํ•ด๋‹น ๋‹จ๊ณ„์—์„œ ๋ชจ๋ธ์— ๋Œ€ํ•ด ๊ณ„์‚ฐ๋œ ์ธก์ •ํ•ญ๋ชฉ์ด ์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ์œผ๋กœ ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค. ์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ์— ๋Œ€ํ•œ log_metrics API๊ฐ€ ์ด ๋ชฉ์ ์œผ๋กœ ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค.

Python

from typing import Dict

from google.cloud import aiplatform


def log_metrics_sample(
    experiment_name: str,
    run_name: str,
    metrics: Dict[str, float],
    project: str,
    location: str,
):
    aiplatform.init(experiment=experiment_name, project=project, location=location)

    aiplatform.start_run(run=run_name)

    aiplatform.log_metrics(metrics)

  • experiment_name: ์‹คํ—˜ ์ด๋ฆ„์„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค. ์„น์…˜ ํƒ์ƒ‰ ๋ฉ”๋‰ด์—์„œ ์‹คํ—˜์„ ์„ ํƒํ•˜๋ฉด Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ๋ชฉ๋ก์„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • run_name: ์‹คํ–‰ ์ด๋ฆ„์„ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค(start_run ์ฐธ์กฐ).
  • metric: ์ธก์ •ํ•ญ๋ชฉ ํ‚ค-๊ฐ’ ์Œ์ž…๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค๋ฉด {'learning_rate': 0.1}์ž…๋‹ˆ๋‹ค.
  • project: ํ”„๋กœ์ ํŠธ ID ์ด๋Ÿฌํ•œ ํ•ญ๋ชฉ์€ Google Cloud ์ฝ˜์†” ์‹œ์ž‘ ํŽ˜์ด์ง€์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • location: ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์œ„์น˜ ๋ชฉ๋ก์„ ์ฐธ์กฐํ•˜์„ธ์š”.

์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ

์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ์„ ๋กœ๊น…ํ•˜๋ ค๋ฉด Vertex AI ์‹คํ—˜์—์„œ ์ง€์› Vertex AI ํ…์„œ๋ณด๋“œ ์ธ์Šคํ„ด์Šค๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ ๋กœ๊น…์˜ ๋ฐฑ์—… Vertex AI ํ…์„œ๋ณด๋“œ ๋ฆฌ์†Œ์Šค ํ• ๋‹น

log_time_series_metrics๋ฅผ ํ†ตํ•ด ๋กœ๊น…๋œ ๋ชจ๋“  ์ธก์ •ํ•ญ๋ชฉ์€ ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ์œผ๋กœ ์ €์žฅ๋ฉ๋‹ˆ๋‹ค. Vertex AI ํ…์„œ๋ณด๋“œ๋Š” ์ง€์› ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ ์ €์žฅ์†Œ์ž…๋‹ˆ๋‹ค.

experiment_tensorboard๋Š” ์‹คํ—˜ ๋ฐ ์‹คํ—˜ ์‹คํ–‰ ์ˆ˜์ค€ ๋ชจ๋‘์—์„œ ์„ค์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์‹คํ–‰ ์ˆ˜์ค€์—์„œ experiment_tensorboard๋ฅผ ์„ค์ •ํ•˜๋ฉด ์‹คํ—˜ ์ˆ˜์ค€์˜ ์„ค์ •์ด ์žฌ์ •์˜๋ฉ๋‹ˆ๋‹ค. ์‹คํ–‰์—์„œ experiment_tensorboard๋ฅผ ์„ค์ •ํ•œ ๋‹ค์Œ์—๋Š” ์‹คํ–‰์˜ experiment_tensorboard๋ฅผ ๋ณ€๊ฒฝํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.

  • ์‹คํ—˜ ์ˆ˜์ค€์—์„œ experiment_tensorboard๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค.
      aiplatform.init(experiment='my-experiment',
                   experiment_tensorboard='projects/.../tensorboard/my-tb-resource')
  • ์‹คํ–‰ ์ˆ˜์ค€์—์„œ experiment_tensorboard๋ฅผ ์„ค์ •ํ•ฉ๋‹ˆ๋‹ค. ์ฐธ๊ณ : ์‹คํ—˜ ์ˆ˜์ค€์—์„œ ์„ค์ •์„ ์žฌ์ •์˜ํ•ฉ๋‹ˆ๋‹ค.
      aiplatform.start_run(run_name='my-other-run',
                        tensorboard='projects/.../.../other-resource')
    aiplatform.log_time_series_metrics(...)

Python

from typing import Dict, Optional

from google.cloud import aiplatform
from google.protobuf import timestamp_pb2


def log_time_series_metrics_sample(
    experiment_name: str,
    run_name: str,
    metrics: Dict[str, float],
    step: Optional[int],
    wall_time: Optional[timestamp_pb2.Timestamp],
    project: str,
    location: str,
):
    aiplatform.init(experiment=experiment_name, project=project, location=location)

    aiplatform.start_run(run=run_name, resume=True)

    aiplatform.log_time_series_metrics(metrics=metrics, step=step, wall_time=wall_time)

  • experiment_name: ์‹คํ—˜ ์ด๋ฆ„์„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค. ์„น์…˜ ํƒ์ƒ‰ ๋ฉ”๋‰ด์—์„œ ์‹คํ—˜์„ ์„ ํƒํ•˜๋ฉด Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ๋ชฉ๋ก์„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • run_name: ์‹คํ–‰ ์ด๋ฆ„์„ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค(start_run ์ฐธ์กฐ).
  • metrics: ํ‚ค๊ฐ€ ์ธก์ •ํ•ญ๋ชฉ ์ด๋ฆ„์ด๊ณ  ๊ฐ’์ด ์ธก์ •ํ•ญ๋ชฉ ๊ฐ’์ธ ์‚ฌ์ „์ž…๋‹ˆ๋‹ค.
  • step: ์„ ํƒ์‚ฌํ•ญ. ์‹คํ–‰ ๋‚ด์—์„œ ์ด ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ์˜ ๋‹จ๊ณ„ ์ƒ‰์ธ์ž…๋‹ˆ๋‹ค.
  • wall_time: ์„ ํƒ์‚ฌํ•ญ. ์ด ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ๊ฐ€ ์ตœ์ข… ์‚ฌ์šฉ์ž์— ์˜ํ•ด ์ƒ์„ฑ๋œ ์‹œ๊ฐ„ ํƒ€์ž„์Šคํƒฌํ”„์ž…๋‹ˆ๋‹ค. ์ œ๊ณตํ•˜์ง€ ์•Š์œผ๋ฉด time.time()์˜ ๊ฐ’์„ ๊ธฐ๋ฐ˜์œผ๋กœ wall_time์ด ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค.
  • project: ํ”„๋กœ์ ํŠธ ID ์ด๋Ÿฌํ•œ ํ•ญ๋ชฉ์€ Google Cloud ์ฝ˜์†” ์‹œ์ž‘ ํŽ˜์ด์ง€์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • location: ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์œ„์น˜ ๋ชฉ๋ก์„ ์ฐธ์กฐํ•˜์„ธ์š”.

๋‹จ๊ณ„ ๋ฐ walltime

log_time_series_metrics API๋Š” ์„ ํƒ์ ์œผ๋กœ step ๋ฐ walltime์„ ํ—ˆ์šฉํ•ฉ๋‹ˆ๋‹ค.

  • step: ์„ ํƒ์‚ฌํ•ญ. ์‹คํ–‰ ๋‚ด์—์„œ ์ด ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ์˜ ๋‹จ๊ณ„ ์ƒ‰์ธ์ž…๋‹ˆ๋‹ค. ์ œ๊ณตํ•˜์ง€ ์•Š์œผ๋ฉด ์ด๋ฏธ ๋กœ๊น…๋œ ๋ชจ๋“  ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ ์ค‘ ์ตœ์‹  ๋‹จ๊ณ„์— ๋Œ€ํ•œ ์ฆ๋ถ„์ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค. ์ œ๊ณต๋œ ์ธก์ •ํ•ญ๋ชฉ ํ‚ค์— ๋Œ€ํ•ด ๋‹จ๊ณ„๊ฐ€ ์กด์žฌํ•˜๋ฉด ํ•ด๋‹น ๋‹จ๊ณ„๋ฅผ ๋ฎ์–ด์”๋‹ˆ๋‹ค.
  • wall_time: ์„ ํƒ์‚ฌํ•ญ. ๋กœ๊น…๋œ ์ธก์ •ํ•ญ๋ชฉ์˜ epoch ์ดํ›„ ์‹œ๊ฐ„(์ดˆ)์ž…๋‹ˆ๋‹ค. ์ œ๊ณต๋˜์ง€ ์•Š์€ ๊ฒฝ์šฐ ๊ธฐ๋ณธ๊ฐ’์€ Python์˜ time.time์ž…๋‹ˆ๋‹ค.

์˜ˆ:

aiplatform.log_time_series_metrics({"mse": 2500.00, "rmse": 50.00})
ํŠน์ • ๋‹จ๊ณ„์— ๋กœ๊น…
aiplatform.log_time_series_metrics({"mse": 2500.00, "rmse": 50.00}, step=8)
wall_time ํฌํ•จ
aiplatform.log_time_series_metrics({"mse": 2500.00, "rmse": 50.00}, step=10)

๋งค๊ฐœ๋ณ€์ˆ˜

๋งค๊ฐœ๋ณ€์ˆ˜๋Š” ์‹คํ–‰์„ ๊ตฌ์„ฑํ•˜๊ณ , ์‹คํ–‰ ๋™์ž‘์„ ์กฐ์ •ํ•˜๊ณ , ์‹คํ–‰ ๊ฒฐ๊ณผ์— ์˜ํ–ฅ์„ ์ฃผ๋Š” ํ‚ค๊ฐ€ ์ง€์ •๋œ ์ž…๋ ฅ ๊ฐ’์ž…๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด ํ•™์Šต๋ฅ , ๋“œ๋กญ์•„์›ƒ ๋น„์œจ, ํ•™์Šต ๋‹จ๊ณ„ ์ˆ˜๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค. log_params ๋ฉ”์„œ๋“œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ ๋กœ๊น…ํ•ฉ๋‹ˆ๋‹ค.

Python

from typing import Dict, Union

from google.cloud import aiplatform


def log_params_sample(
    experiment_name: str,
    run_name: str,
    params: Dict[str, Union[float, int, str]],
    project: str,
    location: str,
):
    aiplatform.init(experiment=experiment_name, project=project, location=location)

    aiplatform.start_run(run=run_name, resume=True)

    aiplatform.log_params(params)

aiplatform.log_params({"learning_rate": 0.01, "n_estimators": 10})
  • experiment_name: ์‹คํ—˜ ์ด๋ฆ„์„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค. ์„น์…˜ ํƒ์ƒ‰ ๋ฉ”๋‰ด์—์„œ ์‹คํ—˜์„ ์„ ํƒํ•˜๋ฉด Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ๋ชฉ๋ก์„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • run_name: ์‹คํ–‰ ์ด๋ฆ„์„ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค(start_run ์ฐธ์กฐ).
  • params: ๋งค๊ฐœ๋ณ€์ˆ˜ ํ‚ค-๊ฐ’ ์Œ์ž…๋‹ˆ๋‹ค. ์˜ˆ: {'accuracy': 0.9}(log_params ์ฐธ์กฐ). ์‹œ์ž‘ ํŽ˜์ด์ง€.
  • location: ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์œ„์น˜ ๋ชฉ๋ก์„ ์ฐธ์กฐํ•˜์„ธ์š”.

๋ถ„๋ฅ˜ ์ธก์ •ํ•ญ๋ชฉ

์š”์•ฝ ์ธก์ •ํ•ญ๋ชฉ๊ณผ ์‹œ๊ณ„์—ด ์ธก์ •ํ•ญ๋ชฉ ์™ธ์— ํ˜ผ๋™ ํ–‰๋ ฌ ๋ฐ ROC ๊ณก์„ ์ด ์ผ๋ฐ˜์ ์œผ๋กœ ์‚ฌ์šฉ๋˜๋Š” ์ธก์ •ํ•ญ๋ชฉ์ž…๋‹ˆ๋‹ค. log_classification_metrics API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Vertex AI Experiments์— ๋กœ๊น…ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

Python

from typing import List, Optional

from google.cloud import aiplatform


def log_classification_metrics_sample(
    experiment_name: str,
    run_name: str,
    project: str,
    location: str,
    labels: Optional[List[str]] = None,
    matrix: Optional[List[List[int]]] = None,
    fpr: Optional[List[float]] = None,
    tpr: Optional[List[float]] = None,
    threshold: Optional[List[float]] = None,
    display_name: Optional[str] = None,
) -> None:
    aiplatform.init(experiment=experiment_name, project=project, location=location)

    aiplatform.start_run(run=run_name, resume=True)

    aiplatform.log_classification_metrics(
        labels=labels,
        matrix=matrix,
        fpr=fpr,
        tpr=tpr,
        threshold=threshold,
        display_name=display_name,
    )

  • experiment_name: ์‹คํ—˜ ์ด๋ฆ„์„ ์ž…๋ ฅํ•ฉ๋‹ˆ๋‹ค. ์„น์…˜ ํƒ์ƒ‰ ๋ฉ”๋‰ด์—์„œ ์‹คํ—˜์„ ์„ ํƒํ•˜๋ฉด Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ๋ชฉ๋ก์„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • run_name: ์‹คํ–‰ ์ด๋ฆ„์„ ์ง€์ •ํ•ฉ๋‹ˆ๋‹ค(start_run ์ฐธ์กฐ).
  • project: ํ”„๋กœ์ ํŠธ ID ์ด๋Ÿฌํ•œ ํ•ญ๋ชฉ์€ Google Cloud ์ฝ˜์†” ์‹œ์ž‘ ํŽ˜์ด์ง€์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
  • location: ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์œ„์น˜ ๋ชฉ๋ก์„ ์ฐธ์กฐํ•˜์„ธ์š”.
  • labels: ํ˜ผ๋™ ํ–‰๋ ฌ์˜ ๋ผ๋ฒจ ์ด๋ฆ„ ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค. 'matrix'๊ฐ€ ์„ค์ •๋œ ๊ฒฝ์šฐ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
  • matrix: ํ˜ผ๋™ ํ–‰๋ ฌ์˜ ๊ฐ’์ž…๋‹ˆ๋‹ค. 'labels'๊ฐ€ ์„ค์ •๋œ ๊ฒฝ์šฐ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
  • fpr: ROC ๊ณก์„ ์˜ ๊ฑฐ์ง“์–‘์„ฑ๋ฅ  ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค. 'tpr' ๋˜๋Š” 'thresholds'๊ฐ€ ์„ค์ •๋œ ๊ฒฝ์šฐ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
  • tpr: ROC ๊ณก์„ ์˜ ์ฐธ์–‘์„ฑ๋ฅ  ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค. 'fpr' ๋˜๋Š” 'thresholds'๊ฐ€ ์„ค์ •๋œ ๊ฒฝ์šฐ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
  • threshold: ROC ๊ณก์„ ์˜ ์ž„๊ณ—๊ฐ’ ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค. 'fpr' ๋˜๋Š” 'tpr'์ด ์„ค์ •๋œ ๊ฒฝ์šฐ ์„ค์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
  • display_name: ๋ถ„๋ฅ˜ ์ธก์ •ํ•ญ๋ชฉ ์•„ํ‹ฐํŒฉํŠธ์˜ ์‚ฌ์šฉ์ž ์ •์˜ ์ด๋ฆ„์ž…๋‹ˆ๋‹ค.

Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ์‹คํ–‰ ๋ชฉ๋ก ๋ณด๊ธฐ

  1. Google Cloud ์ฝ˜์†”์—์„œ ์‹คํ—˜ ํŽ˜์ด์ง€๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.
    ์‹คํ—˜์œผ๋กœ ์ด๋™
    ์‹คํ—˜ ๋ชฉ๋ก์ด ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค.
  2. ํ™•์ธํ•  ์‹คํ—˜์„ ์„ ํƒํ•ฉ๋‹ˆ๋‹ค.
    ์‹คํ–‰ ๋ชฉ๋ก์ด ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค.

Vertex AI ์‹คํ—˜ ๋ชฉ๋ก ํŽ˜์ด์ง€
์ž์„ธํ•œ ๋‚ด์šฉ์€ ์‹คํ–‰ ๋น„๊ต ๋ฐ ๋ถ„์„์„ ์ฐธ์กฐํ•˜์„ธ์š”.

๋‹ค์Œ ๋‹จ๊ณ„

๋…ธํŠธ๋ถ ํŠœํ† ๋ฆฌ์–ผ

๋ธ”๋กœ๊ทธ ๊ฒŒ์‹œ๋ฌผ