Transfers data from external data sources to a Google Cloud Storage bucket or between Google Cloud Storage buckets.

Represents an agent pool.

AWS access key (see AWS Security Credentials). For information on our data retention policy for user credentials, see User credentials.

An AwsS3CompatibleData resource.

An AwsS3Data resource can be a data source, but not a data sink. In an AwsS3Data resource, an object's name is the S3 object's key name.

An AzureBlobStorageData resource can be a data source, but not a data sink. An AzureBlobStorageData resource represents one Azure container. The storage account determines the Azure endpoint. In an AzureBlobStorageData resource, a blobs's name is the Azure Blob Storage blob's key name.

Azure credentials For information on our data retention policy for user credentials, see User credentials.

Specifies a bandwidth limit for an agent pool.

The request message for Operations.CancelOperation.

Defines the root interface for all clients that generate credentials for calling Google APIs. All clients should implement this interface.

Represents a whole or partial calendar date, such as a birthday. The time of day and time zone are either specified elsewhere or are insignificant. The date is relative to the Gregorian Calendar. This can represent one of the following: * A full date, with non-zero year, month, and day values. * A month and day, with a zero year (for example, an anniversary). * A year on its own, with a zero month and a zero day. * A year and month, with a zero day (for example, a credit card expiration date). Related types: * google.type.TimeOfDay * google.type.DateTime * google.protobuf.Timestamp

A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }

An entry describing an error that has occurred.

A summary of errors by error code, plus a count and sample error log entries.

Specifies the Event-driven transfer options. Event-driven transfers listen to an event stream to transfer updated files.

The identity of an Azure application through which Storage Transfer Service can authenticate requests using Azure workload identity federation. Storage Transfer Service can issue requests to Azure Storage through registered Azure applications, eliminating the need to pass credentials to Storage Transfer Service directly. To configure federated identity, see Configure access to Microsoft Azure Storage.

In a GcsData resource, an object's name is the Cloud Storage object's name and its "last modification time" refers to the object's updated property of Cloud Storage objects, which changes when the content or the metadata of the object is updated.

Google service account

An HdfsData resource specifies a path within an HDFS entity (e.g. a cluster). All cluster-specific settings, such as namenodes and ports, are configured on the transfer agents servicing requests, so HdfsData only contains the root path to the data in our transfer.

An HttpData resource specifies a list of objects on the web to be transferred over HTTP. The information of the objects to be transferred is contained in a file referenced by a URL. The first line in the file must be "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines specify the information of the list of objects, one object per list entry. Each entry has the following tab-delimited fields: * HTTP URL — The location of the object. * Length — The size of the object in bytes. * MD5 — The base64-encoded MD5 hash of the object. For an example of a valid TSV file, see Transferring data from URLs. When transferring data based on a URL list, keep the following in mind: * When an object located at http(s)://hostname:port/ is transferred to a data sink, the name of the object at the data sink is /. * If the specified size of an object does not match the actual size of the object fetched, the object is not transferred. * If the specified MD5 does not match the MD5 computed from the transferred bytes, the object transfer fails. * Ensure that each URL you specify is publicly accessible. For example, in Cloud Storage you can [share an object publicly] (/storage/docs/cloud-console#_sharingdata) and get a link to it. * Storage Transfer Service obeys robots.txt rules and requires the source HTTP server to support Range requests and to return a Content-Length header in each response. * ObjectConditions have no effect when filtering objects to transfer.

Response from ListAgentPools.

The response message for Operations.ListOperations.

Response from ListTransferJobs.

Specifies the logging behavior for transfer operations. Logs can be sent to Cloud Logging for all transfer types. See Read transfer logs for details.

Specifies the metadata options for running a transfer.

Specification to configure notifications published to Pub/Sub. Notifications are published to the customer-provided topic using the following PubsubMessage.attributes: * "eventType": one of the EventType values * "payloadFormat": one of the PayloadFormat values * "projectId": the project_id of the TransferOperation * "transferJobName": the transfer_job_name of the TransferOperation * "transferOperationName": the name of the TransferOperation The PubsubMessage.data contains a TransferOperation resource formatted according to the specified PayloadFormat.

Conditions that determine which objects are transferred. Applies only to Cloud Data Sources such as S3, Azure, and Cloud Storage. The "last modification time" refers to the time of the last change to the object's content or metadata — specifically, this is the updated property of Cloud Storage objects, the LastModified field of S3 objects, and the Last-Modified header of Azure blobs. For S3 objects, the LastModified value is the time the object begins uploading. If the object meets your "last modification time" criteria, but has not finished uploading, the object is not transferred. See Transfer from Amazon S3 to Cloud Storage for more information. Transfers with a PosixFilesystem source or destination don't support ObjectConditions.

This resource represents a long-running operation that is the result of a network API call.

Request passed to PauseTransferOperation.

A POSIX filesystem resource.

Additional options for StorageTransfer#projectsAgentPoolsCreate.

Additional options for StorageTransfer#projectsAgentPoolsList.

Additional options for StorageTransfer#projectsAgentPoolsPatch.

Specifies the configuration for a cross-bucket replication job. Cross-bucket replication copies new or updated objects from a source Cloud Storage bucket to a destination Cloud Storage bucket. Existing objects in the source bucket are not copied by a new cross-bucket replication job.

Request passed to ResumeTransferOperation.

Request passed to RunTransferJob.

S3CompatibleMetadata contains the metadata fields that apply to the basic types of S3-compatible data providers.

Transfers can be scheduled to recur or to run just once.

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the API Design Guide.

Represents a time of day. The date and time zone are either not significant or are specified elsewhere. An API may choose to allow leap seconds. Related types are google.type.Date and google.protobuf.Timestamp.

A collection of counters that report the progress of a transfer operation.

This resource represents the configuration of a transfer job that runs periodically.

Additional options for StorageTransfer#transferJobsDelete.

Additional options for StorageTransfer#transferJobsGet.

Additional options for StorageTransfer#transferJobsList.

Specifies where the manifest is located.

A description of the execution of a transfer.

Additional options for StorageTransfer#transferOperationsList.

TransferOptions define the actions to be performed on objects in a transfer.

Configuration for running a transfer.

Request passed to UpdateTransferJob.