This may take a few minutes. When creating a new sync relationship, you can choose an S3 bucket that is not associated with your AWS account. Define the number of times that Cloud Sync should retry to sync a file before skipping it. You no longer need to manually enter commands to install the data broker in Azure. Learn how to install the data broker on an existing Linux host. This does not break the sync relationship. If you enable this option, you also need to change a parameter in the local.json file on the data broker. We changed how you interact with data brokers by emphasizing data broker groups. You can now view and select objects from S3 buckets that belong to other AWS accounts if they are shared with your account. Learn more about copying ACLs between SMB/CIFS shares. For example, Cloud Sync might recommend how to optimize a sync relationship for Azure NetApp Files. You then choose the target location for the new sync relationship. Credentials are now locally encrypted on the data broker machine. ONTAP S3 is now supported in a sync relationship with StorageGRID. Cloud Sync now supports multi-tenancy through Cloud Central accounts. Thanks for your patience. This update includes performance improvements and usability enhancements to the data broker creation wizard. Go to the Cloud Sync section in the Cloud Manager documentation, Learn more about defining a unified configuration, Learn how to copy ACLs between NFS servers, Learn more about installing the data broker in AWS, Learn more about installing the data broker in Google Cloud, Learn more about data-in-flight encryption, Learn more about ONTAP support for Amazon S3, Review Cloud Sync requirements for ONTAP S3 Storage, Learn more about copying ACLs between SMB servers, Learn more about copying ACLs between SMB servers and review the requirements, Learn how to activate support and contact NetApp technical support, Learn more about installing the data broker in Azure, View the list of supported sync relationships, Review details about using your own IAM role, Learn how to deploy the data broker in GCP, Learn how to install the data broker on an existing Linux host, Learn more about copying ACLs between SMB/CIFS shares. Cloud Sync can now preserve access control lists (ACLs) between a source SMB/CIFS share and a target SMB/CIFS share when creating a new sync relationship. Youll need to add the token and account ID to the Authorization header when making API calls. uid, gid, and mode: Permission flags for Linux. Cloud Sync now supports copying ACLs from object storage to SMB shares. Review details about using your own IAM role. Cloud Sync now supports installing the data broker in Microsoft Azure, which enables you to sync data in and out of Azure. Youll find these recommendations on the Sync Relationships dashboard. You can now accelerate the performance of a sync relationship by adding an additional data broker to the relationship. We redesigned the table view on the Dashboard for ease of use. The ID field represents the last message that the server sent. The target can be an NFS or SMB server. Learn how to activate support and contact NetApp technical support. Creating a sync relationship from SFTP to Amazon S3 is now supported in the user interface. Finally, you can also delete data broker groups. From within Data Sense, you can select the source files that youd like to sync to a target location using Cloud Sync. You can now update the proxy configuration for a data broker. When creating a sync relationship, you can now add tags to the object storage target in a sync relationship. You can add the name when you create the relationship and any time after. You will need to register your support serial number to activate support before you can contact NetApp technical support. For example, when you create a new sync relationship, you select the data broker group to use with the relationship, rather than a specific data broker. You still have the option to use a CloudFormation template, if youd rather not provide an access key. The deployment of the AWS data broker has been simplified. We enhanced how Cloud Sync copies access control lists (ACLs) between SMB servers. The standalone Cloud Sync service has been retired. You might use this option if your organization has strict security policies. It must be a regular connection string, not a shared access signature (SAS). Thanks for your patience. If there are any changes, then it syncs those files and directories.
When you sync data to or from Amazon S3, you can choose whether to use an S3 private link. The only response from the server is HTTP status 202. You can now subscribe to Cloud Sync from Azure where you can pay as-you-go with hourly rates, or pay up front for a year. These credentials are now encrypted using HashiCorp Vault. A new sync relationship setting enables you to sync files based on the last modified date. If you extended a Cloud Sync license that you purchased from NetApp, you can add the license again to refresh the expiration date. After you initiate a data sync from Cloud Data Sense, all of the source information is contained in a single step and only requires you to enter a few key details. When you create a sync relationship where Azure Blob is the target, Cloud Sync now enables you to copy tags to the Azure Blob container: On the Settings page, you can use the Copy for Objects setting to copy tags from the source to the Azure Blob container. You can now deploy a data broker in AWS using your own IAM role, rather than the IAM role that Cloud Sync creates for you. When you deploy a data broker in Google Cloud, you can now choose whether to enable or disable a public IP address for the virtual machine instance. This setting is available only when syncing from an S3 bucket to S3, Google Cloud Storage, Azure Blob storage, StorageGRID, or IBM Storage. This setting enables you to define whether Cloud Sync should cancel a data sync if the sync hasnt completed in the specified number of hours or days. Cloud Sync now provides additional stats about the last data sync for each sync relationship: How many directories and files were scanned, How many directories and files were marked for copy and marked for deletion.
This enhancement can help if your security policies require you to update credentials on a periodic basis. Learn more about changing the settings for a sync relationship. You must now select an NFS version or SMB version when setting up a new sync relationship for an NFS or SMB server. You no longer need to use a CloudFormation template to install the data broker in AWS. You can now sync data from an NFS server or SMB server to Azure Data Lake Storage Gen2. Cloud Sync can now copy object metadata and tags between object-based storage when you create a sync relationship and enable a setting. Cloud Sync syncs data at the next scheduled time. Cloud Sync now supports ONTAP S3 Storage to ONTAP S3 Storage sync relationships. Then you can select the Blob container. Improved performance when copying large NFS files in a low latency environment. A data broker that uses a proxy server can now be part of a sync relationship that includes an Azure Blob container. Learn how to update credentials. You can disable tagging of S3 objects, if its not desired in your environment. Starting today, this feature works with any type of data broker: the AWS, Azure, Google Cloud Platform, or on-prem data broker. Heres the Settings page when creating a new relationship: Heres where to access the Settings option for an existing relationship: We improved the speed of the user interface. When creating a sync relationship between an Azure Blob container and an NFS or SMB server, you must now provide the storage account connection string. EFS to S3 and S3 to EFS sync relationships are now supported. EFS to NFS and NFS to EFS sync relationships are now supported. You simply need to check a box when you create a relationship or after you create a relationship. On the Directories page, select a top-level directory or subdirectory. Cloud Sync now supports sync relationships between ONTAP S3 Storage and a Google Cloud Storage bucket from the user interface. Cloud Sync can now copy access control lists (ACLs) when setting up a sync relationship from a source SMB share to object storage (except for ONTAP S3). You can now sync NFS data from Azure NetApp Files to Azure NetApp Files using data-in-flight encryption. Cloud Sync now shows you details about failures that occurred in the most recent data sync. The following steps provide an example of how to create a sync relationship from an NFS server to an S3 bucket. You can schedule a relationship to sync data as often as every 1 minute. Choose whether Cloud Sync should compare certain attributes when determining whether a file or directory has changed and should be synced again. Learn more about defining a unified configuration. On the Bucket Setup page, set up the bucket: Choose whether to enable S3 bucket encryption and then select an AWS KMS key, enter the ARN of a KMS key, or select AES-256 encryption. Before you can create some types of sync relationships, youll first need to create a working environment in Cloud Manager. To refresh the token, you need to call the API from step 1 again. You can enable notifications for successful data syncs, failed data syncs, and canceled data syncs. View the supported storage classes. Data broker connection errors now display on the Dashboard. You can now contact NetApp technical support to get help with Cloud Sync. You can choose to enable or disable Cloud Sync from comparing the following attributes: mtime: The last modified time for a file. Sync relationship settings were moved to a new page when setting up a relationship and when editing a relationship. Open the file and update it as follows: Choose to delete files from the target location, if they were deleted from the source. We simplified the installation steps for the on-premises data broker. Learn more about the Copy for Objects setting. View the list of supported sync relationships. Learn more about copying ACLs between SMB servers. You can now download data broker logs at any time. Cloud Sync now provides recommendations when it identifies ways for you to accelerate or optimize a sync relationship. If a user changes the metadata on the source, Cloud Sync copies this object in the next sync, but if a user changes the tags on the source (and not the data itself), Cloud Sync doesnt copy the object in the next sync. This sync relationship was previously supported with the API only. Cloud Sync now gives you the option to enter an AWS access key and then fill out a quick deployment wizard. You can now restart a data broker from Cloud Sync. For NFS, you can select version 3, 4.1, or 4.2. Cloud Sync then shows you the right commands to use. On the Tags/Metadata page, choose whether to save a key-value pair as a tag on all files transferred to the S3 bucket or to assign a metadata key-value pair on all files. Cloud Sync doesnt support copying ACLs from object storage to SMB shares. Specify file extensions to exclude from the sync by typing the file extension and pressing Enter. The Cloud Sync capabilities that are available through the web UI are also available through the RESTful API. Cloud Sync supports the following storage classes: You can now view the errors found in reports and you can delete the last report or all reports. Cloud Sync now supports additional sync relationships for Box when using the Cloud Sync API: Learn how to set up a sync relationship using the API. The following video provides a short demo: Choose to sync all files regardless of their size or just files that are in a specific size range. View the list of supported sync relationships. NFS 4.1 is now supported with Azure NetApp Files. If S3 is the source, filter by size will be active only on copy events (not on delete events). For example, heres a recommendation for an Azure NetApp Files relationship: The deployment of the Google Cloud data broker has been simplified. The API response includes an "expires_in" field that states when the token expires. This may take a few minutes. Previously, you could only specify relationship metadata. You can now choose the Glacier Instant Retrieval storage class when Amazon S3 is the target in a sync relationship. You would simply replace
If errors occur, you can use the failure reason to correct the issue. You can schedule a relationship to sync data as often as every 1 minute. This setting helps avoid copying partial changes to files that frequently change. You no longer need to manually enter commands to install the data broker in GCP. You now choose whether the data broker should have AWS access, GCP access, or both, and the type of proxy configuration that you have. Cloud Sync is now integrated with NetApp Cloud Central, which enables centralized user authentication. This advanced setting enables you to choose whether Cloud Sync should compare certain attributes when determining whether a file or directory has changed and should be synced again. Two new settings are available for sync relationships: When you enable this setting, Cloud Sync deletes files from the source location after it copies the files to the target location. A new Sync Timeout setting is now available for sync relationships. When an SMB server is the source, this setting enables you to sync files that were created after a specific date, before a specific date, or between a specific time range. For Azure and Google Cloud Storage, only the metadata option is available. A few bugs were fixed. These stats enable you to see more details about the most recent data sync. A new Compare by setting is now available for each sync relationship. On the NFS Server page, enter the IP address or fully qualified domain name of the NFS server that you want to sync to AWS.
Cloud Sync now copies metadata between Google Cloud Storage and S3 providers (AWS S3, StorageGRID, and IBM Cloud Object Storage). Define whether Cloud Sync should cancel a data sync if the sync hasnt completed in the specified number of hours or days. On the Data Broker Group page, follow the prompts to create a data broker virtual machine in AWS, Azure, or Google Cloud Platform, or to install the data broker software an existing Linux host. Cloud Sync now supports sync relationships between ONTAP S3 Storage and SMB servers: Cloud Sync now enables you to unify a data broker groups configuration directly from the user interface. A new uninstall script for the data broker removes packages and directories that were created when the data broker was installed. Choose all files regardless of their last modified date, files modified after a specific date, before a specific date, or between a time range. When you create a sync relationship, Cloud Sync might be unable to retrieve the shares for an SMB server. The Continuous Sync setting is now supported from a source Google Cloud Storage bucket to a cloud storage target. This setting enables you to choose whether to receive Cloud Sync notifications in Cloud Managers Notification Center. If Cloud Sync is unable to retrieve the exports, click Add Export Manually and enter the name of an NFS export. The default is 24 hours. Cloud Sync now identifies when a data broker isnt running the latest software release. This makes it easy to securely transfer data across subnets or regions. In the previous release, we introduced Cloud Sync integration with Cloud Data Sense. For example, type log or .log to exclude *.log files. For more details, refer to the following pages: Installing the data broker on a Linux host.
After the initial copy, the service syncs any changed data every 24 hours. In the Manage Data Brokers tab, we also show the number of sync relationships that a data broker group is managing. Create an Amazon FSx for ONTAP working environment, Setting up and discovering Azure NetApp Files, Launching Cloud Volumes ONTAP in Google Cloud, Adding existing Cloud Volumes ONTAP systems. When you create a sync relationship, you can now refresh the list of directories or shares for the source or target. Two new sync relationships are supported: IBM Cloud Object Storage to IBM Cloud Object Storage. Cloud Sync now shows the storage class used with an S3 sync relationship, even when its the Standard storage class.
Thanks for your patience. You can now specify whether files modified prior to the scheduled sync should be excluded. When syncing data, Cloud Sync copies ACLs between NFS servers that use NFS versions 4.0, 4.1, or 4.2. Cloud Sync doesnt sync any metadata that includes special characters. Theres no need to rescan the source at scheduled intervals. When you create a sync relationship to or from Google Cloud Storage, Cloud Sync no longer prompts you to provide a project ID, client email, and private key for a Cloud Storage service account. Just follow the prompts in Cloud Sync to deploy a virtual machine in Google Cloud Platform that runs the data broker software. Heres an example: Grouping data brokers together can help improve the performance of sync relationships. You can now set up sync relationships to sync data between: An AWS S3 bucket and an Azure Blob container, An AWS S3 bucket and a Google Cloud Storage bucket. Cloud Sync now supports sync relationships from SFTP to S3 by using the API. To get started with the Cloud Sync API, you need to obtain a user token and your Cloud Central account ID. View the list of supported sync relationships, Learn more about the Continuous Sync setting, Learn more about changing the settings for a sync relationship, Learn how to set up a sync relationship using the API, Learn how to deploy a data broker in Google Cloud, Learn more about the Copy for Objects setting, Learn more about using a HashiCorp Vault with a data broker, Learn more about creating and viewing reports to tune your configuration. Learn more about installing the data broker in Azure. On the AWS S3 Bucket page, select a bucket: Drill down to select an existing folder within the bucket or to select a new folder that you create inside the bucket. Click View details about sync failures (as shown in the image above) and youll see a page similar to the following: Click Download Data Broker Logs to download logs that can help with troubleshooting. Adding tags is supported with Amazon S3, Azure Blob, Google Cloud Storage, IBM Cloud Object Storage, and StorageGRID. Just click the download button thats available from each sync relationship. Learn more about paying for sync relationships. Even if you uncheck these attributes, Cloud Sync still compares the source to the target by checking the paths, file sizes, and file names. You can enable notifications for successful data syncs, failed data syncs, and canceled data syncs. Learn more about this setting. Learn how to create a sync relationship for Amazon FSx for ONTAP. Learn how to start a sync relationship from Cloud Data Sense. The steps below provide an example that shows how to set up a sync relationship from an NFS server to an S3 bucket. Cloud Sync now supports NFSv4 ACLs. Thanks for your patience. Or it might recommend that you accelerate a sync relationship if many large directories are being synced or if there are too many relationships per data broker. S3 to S3 sync relationships are now supported. Note that the IAM role associated with your data broker will need the following permissions to use this setting: These permissions are automatically added to any new data brokers that you create. Enable this option to copy object storage metadata and tags. Enables you to choose whether to receive Cloud Sync notifications in Cloud Managers Notification Center. You should now access Cloud Sync directly from Cloud Manager where all of the same features and functionality are available. A new reporting feature provides information that you can use with the help of NetApp personnel to tune a data brokers configuration and improve performance. When you choose the source or target volume for Azure NetApp Files, Cloud Sync now displays a dual-protocol volume no matter which protocol you chose for the sync relationship. You can now filter source objects when setting up a new sync relationship. Google Cloud Documentation: Creating and Managing Service Account Keys. Filtering source objects enables you to define how source files and folders are synced and maintained in the target location. For example: This may take a few minutes. An Azure Blob container is now supported as the source or target in a sync relationship with an NFS or SMB server. If this happens, you can now click Add Share Manually and enter the name of an SMB share.
StorageGRID Webscale is now supported as the source in a sync relationship. Accounts enable multiple users to manage the same sync relationships in an account. Copy ACLs from an SMB server by enabling a setting when you create a relationship or after you create a relationship. Cloud Sync now provides a Microsoft Azure login form and a quick deployment wizard. After logging in to Cloud Manager, you can switch to the Sync tab to view and manage your relationships, just like before. When a Google Cloud Storage bucket is the target in a sync relationship, you can now choose the storage class that you want to use.
You can now change the sync schedule for a relationship to as frequently as 5 minutes. When an SMB server is the source, a new sync relationship setting called Date Created enables you to sync files that were created after a specific date, before a specific date, or between a specific time range. Previously, Cloud Sync would only display the volumes that matched the selected protocol. ONTAP 9.7 supports the Amazon Simple Storage Service (Amazon S3) as a public preview. You can now sync data between two Azure Blob containers. You can access the option when selecting a directory: The following options are available when filtering source objects: Incremental updates from an S3 bucket to an NFS or SMB server are no longer event-driventhey are based on a sync schedule. Learn how to deploy the data broker in GCP. Choose all files regardless of their last modified date, files modified after a specific date, before a specific date, or between a time range. When an Amazon S3 bucket is the target in a sync relationship, you can now enable S3 bucket encryption by selecting an AWS KMS key, entering the ARN of a KMS key, or by selecting AES-256 encryption. For SMB, you can select version 1.0, 2.0, 2.1, or 3.0. Choose to delete files from the source location after Cloud Sync copies the files to the target location. The on-prem data broker is now supported with additional Linux distributions: SUSE Linux Enterprise Server 12.4 SP5 and 15 SP1. Cloud Sync now supports deploying the data broker in the AWS Asia Pacific (Jakarta) region.
When the data broker copies data from the source to the target, it goes through the private link. Azure NetApp Files (SMB) is now supported as the source or target in a sync relationship.