Stay organized with collections
Save and categorize content based on your preferences.
With Private Service Connect, you can create a
Dataproc Metastore service that doesn't use VPC
peering. This page explains what Private Service Connect is
and how to use it as an alternative networking option for
Dataproc Metastore.
How Dataproc Metastore works with VPC
Dataproc Metastore protects its metadata access by only exposing private
IP endpoints. It also restricts connectivity to VMs in your VPC network by
using VPC peering.
Dataproc Metastore requires the following configurations
per region for each VPC network:
As a result, setting up VPC peering and IP address reservations
can be difficult on crowded VPC networks. Similarly, a VPC
network may not have enough peering quota to accommodate additional peering
requests. Both of these limitations can prevent you from creating new
Dataproc Metastore service.
To circumvent these issues, you can use
Dataproc Metastore with Private Service Connect.
How Dataproc Metastore works with Private Service Connect
Private Service Connect provides a private connection to
Dataproc Metastore metadata across VPC networks.
To use Private Service Connect with Dataproc Metastore,
the following configurations are required:
A single address reservation in the subnetwork.
A forwarding rule targeting the service attachment that exposes the Dataproc Metastore
endpoint. The address reservation and forwarding rule are created as a part of
the Dataproc Metastore service create call.
Considerations
Dataproc Metastore services that use Private Service Connect
only support access from VPC networks of the subnetworks specified during the service
creation.
Dataproc Metastore reserves addresses and creates forwarding
rules in each of the specified subnetworks. Each subnetwork has a Thrift
endpoint URI that you can use to access the Dataproc Metastore
metadata endpoint from.
Limitations
Dataproc Metastore services that use the gRPC endpoints protocol
don't support Private Service Connect.
Reverse connectivity is not supported with Private Service Connect.
This means you can't use a Kerberos configuration with Private Service Connect.
You can't dynamically add or remove subnets from a Dataproc Metastore
service that is configured with Private Service Connect. Instead, you
must recreate a service if you'd like to add or remove subnets.
You can't update a Dataproc Metastore service that uses Private Service Connect
to use VPC or the other way around.
Create a Dataproc Metastore service with Private Service Connect
The following instructions demonstrate how to configure Private Service Connect
during service creation.
Console
In the Google Cloud console, open the Dataproc Metastore page:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-28 UTC."],[[["\u003cp\u003eDataproc Metastore can use Private Service Connect as an alternative to VPC peering, offering a private connection to metadata across VPC networks.\u003c/p\u003e\n"],["\u003cp\u003eUsing Private Service Connect with Dataproc Metastore requires a single address reservation in the subnetwork and a forwarding rule targeting the service attachment, and this is automatically created during the service call creation.\u003c/p\u003e\n"],["\u003cp\u003eDataproc Metastore services with Private Service Connect only support access from specified VPC subnetworks and do not allow reverse connectivity, which means Kerberos configuration is not supported.\u003c/p\u003e\n"],["\u003cp\u003ePrivate Service Connect, used by Dataproc Metastore, does not support the gRPC endpoints protocol, nor does it allow dynamic addition or removal of subnetworks, and it is not able to switch back and forth between VPC.\u003c/p\u003e\n"],["\u003cp\u003eTo configure Private Service Connect during Dataproc service creation, one can use the Google Cloud console, gcloud commands, or the REST API, specifying the subnetworks within the 'Network Configuration'.\u003c/p\u003e\n"]]],[],null,["# Private Service Connect with Dataproc Metastore\n\nWith Private Service Connect, you can create a\nDataproc Metastore service that doesn't use [VPC\npeering](/vpc/docs/vpc-peering). This page explains what [Private Service Connect](/vpc/docs/private-service-connect) is\nand how to use it as an alternative networking option for\nDataproc Metastore.\n\nHow Dataproc Metastore works with VPC\n-------------------------------------\n\nDataproc Metastore protects its metadata access by only exposing private\nIP endpoints. It also restricts connectivity to VMs in your VPC network by\nusing VPC peering.\n\nDataproc Metastore requires the following configurations\nper region for each VPC network:\n\n- 1 [peering quota](/vpc/docs/quota)\n- `/17` and `/20` [CIDR](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) blocks\n\nAs a result, setting up VPC peering and IP address reservations\ncan be difficult on crowded VPC networks. Similarly, a VPC\nnetwork may not have enough peering quota to accommodate additional peering\nrequests. Both of these limitations can prevent you from creating new\nDataproc Metastore service.\n\nTo circumvent these issues, you can use\nDataproc Metastore with Private Service Connect.\n\nHow Dataproc Metastore works with Private Service Connect\n---------------------------------------------------------\n\nPrivate Service Connect provides a private connection to\nDataproc Metastore metadata across VPC networks.\n\nTo use Private Service Connect with Dataproc Metastore,\nthe following configurations are required:\n\n- A single address reservation in the subnetwork.\n- A forwarding rule targeting the service attachment that exposes the Dataproc Metastore endpoint. The address reservation and forwarding rule are created as a part of the Dataproc Metastore service create call.\n\n### Considerations\n\n- Dataproc Metastore services that use Private Service Connect\n only support access from VPC networks of the subnetworks specified during the service\n creation.\n\n- Dataproc Metastore reserves addresses and creates forwarding\n rules in each of the specified subnetworks. Each subnetwork has a Thrift\n endpoint URI that you can use to access the Dataproc Metastore\n metadata endpoint from.\n\n### Limitations\n\n- Dataproc Metastore services that use the gRPC endpoints protocol don't support Private Service Connect.\n- Reverse connectivity is not supported with Private Service Connect. This means you can't use a Kerberos configuration with Private Service Connect.\n- You can't dynamically add or remove subnets from a Dataproc Metastore service that is configured with Private Service Connect. Instead, you must recreate a service if you'd like to add or remove subnets.\n- You can't update a Dataproc Metastore service that uses Private Service Connect to use VPC or the other way around.\n\nCreate a Dataproc Metastore service with Private Service Connect\n----------------------------------------------------------------\n\nThe following instructions demonstrate how to configure Private Service Connect\nduring service creation. \n\n### Console\n\n1. In the Google Cloud console, open the Dataproc Metastore page:\n\n [Go to Dataproc Metastore](https://console.cloud.google.com/dataproc/metastore/services)\n2. At the top of the **Dataproc Metastore** page, click **Create**.\n\n The **Create service** page opens.\n3. Configure your service as needed.\n\n4. Under **Network configuration** , click **Make services accessible in multiple VPC subnetworks**.\n\n5. Select the **Subnetworks**. You can specify up to 5 subnetworks.\n\n6. Click **Done**.\n\n7. Click **Submit**.\n\nVerify the service's network configuration:\n\n1. In the Google Cloud console, open the Dataproc Metastore page:\n\n [Go to Dataproc Metastore](https://console.cloud.google.com/dataproc/metastore/services)\n2. On the **Dataproc Metastore** page, click the service name of the\n service want like to view.\n\n The **Service detail** page for that service\n opens.\n3. Under the **Configuration** tab, verify that the details show multiple VPC\n subnetwork URIs.\n\n### gcloud\n\n1. Run the following `gcloud metastore services create` command to\n create a service with Private Service Connect:\n\n ```\n gcloud metastore services create SERVICE \\\n --location=LOCATION \\\n --consumer-subnetworks=\"projects/PROJECT_ID/regions/LOCATION/subnetworks/SUBNET1, projects/PROJECT_ID/regions/LOCATION/subnetworks/SUBNET2\"\n ```\n\n or \n\n ```\n gcloud metastore services create SERVICE \\\n --location=LOCATION \\\n --network-config-from-file=NETWORK_CONFIG_FROM_FILE\n ```\n2. Verify that the creation was successful.\n\n### REST\n\nFollow the API instructions to [create a service](/dataproc-metastore/docs/reference/rest/v1/projects.locations.services/create)\nby using the API Explorer.\n\nIn the `create` request parameters, use the `Network Config` filed to\nconfigure Private Service Connect. You can specify 1 to 5 subnetworks.\n\n\u003cbr /\u003e\n\n```\n \"network_config\": {\n \"consumers\": [\n {\"subnetwork\": \"projects/PROJECT_ID/regions/LOCATION/subnetworks/SUBNET1\"},\n {\"subnetwork\": \"projects/PROJECT_ID/regions/LOCATION/subnetworks/SUBNET2\"}\n ]\n }\n \n```\n\n\u003cbr /\u003e\n\nWhat's next\n-----------\n\n- [VPC Network Peering overview](/vpc/docs/vpc-peering)\n- [Private Service Connect](/vpc/docs/private-service-connect)\n- [Service networking](/dataproc-metastore/docs/networking)"]]
