Arctera Data Insight Administrator's Guide
- Section I. Getting started
- Introduction to Arctera Data Insight administration
- Configuring Data Insight global settings
- About Data Insight licensing
- SQLite WAL mode
- Configuring SMTP server settings
- About scanning and event monitoring
- Monitoring Indexer Node Storage Utilization
- About filtering certain accounts, IP addresses, and paths
- About archiving data
- About Data Insight integration with Data Loss Prevention (DLP)
- Importing sensitive files information through CSV
- Configuring advanced analytics
- About open shares
- About user risk score
- Configuring file groups
- Configuring Workspace data owner policy
- Configuring Management Console settings
- About bulk assignment of custodians
- Configuring Watchlist settings
- Configuring Metadata Framework
- Proof of concept
- Section II. Configuring Data Insight
- Configuring Data Insight product users
- Configuring Data Insight product servers
- About Data Insight product servers
- Adding a new Data Insight server
- Managing Data Insight product servers
- Viewing Data Insight server details
- About node templates
- Adding Portal role to a Data Insight server
- Adding Classification Server role to a Data Insight server
- Assigning Classification Server to a Collector
- Associating a Classification Server pool to a Collector
- Viewing in-progress scans
- Configuring Data Insight services
- Configuring advanced settings
- Monitoring Data Insight jobs
- Rotating the encryption keys
- Viewing Data Insight server statistics
- About automated alerts for patches and upgrades
- Deploying upgrades and patches remotely
- Using the Upload Manager utility
- About migrating storage devices across Indexers
- Viewing the status of a remote installation
- Configuring saved credentials
- Configuring directory service domains
- About directory domain scans
- Adding a directory service domain to Data Insight
- Managing directory service domains
- Fetching users and groups data from NIS+ scanner
- Configuring attributes for advanced analytics
- Deleting directory service domains
- Scheduling scans
- Configuring business unit mappings
- Importing additional attributes for users and user groups
- Configuring containers
- Section III. Configuring native file systems in Data Insight
- Configuring NetApp 7-mode file server monitoring
- About configuring NetApp file server monitoring
- Prerequisites for configuring NetApp file servers
- Credentials required for configuring NetApp filers
- Credentials required for configuring NetApp NFS filers
- Configuring SMB signing
- About FPolicy
- Preparing Data Insight for FPolicy
- Preparing the NetApp filer for Fpolicy
- Preparing the NetApp vfiler for Fpolicy
- Configuring NetApp audit settings for performance improvement
- Preparing a non-administrator domain user on the NetApp filer for Data Insight
- Enabling export of NFS shares on a NetApp file server
- Excluding volumes on a NetApp file server
- Handling NetApp home directories in Data Insight
- Configuring clustered NetApp file server monitoring
- About configuring a clustered NetApp file server
- About configuring FPolicy in Cluster-Mode
- Pre-requisites for configuring clustered NetApp file servers
- Credentials required for configuring a clustered NetApp file server
- Preparing a non-administrator local user on the clustered NetApp filer
- Preparing a non-administrator domain user on a NetApp cluster for Data Insight
- Persistent Store
- Preparing Data Insight for FPolicy in NetApp Cluster-Mode
- Preparing the ONTAP cluster for FPolicy
- About configuring secure communication between Data Insight and cluster-mode NetApp devices
- Enabling export of NFS shares on a NetApp Cluster-Mode file server
- Enabling SSL support for Cluster Mode NetApp auditing
- Configuring EMC Celerra or VNX monitoring
- Configuring EMC Isilon monitoring
- About configuring EMC Isilon filers
- Prerequisites for configuration of Isilon or Unity VSA file server monitoring
- Credentials required for configuring an EMC Isilon cluster
- Configuring audit settings on EMC Isilon cluster using OneFS GUI console
- Configuring audit settings on EMC Isilon cluster using the OneFS CLI
- Configuring Isilon audit settings for performance improvement
- Preparing Arctera Data Insight to receive event notifications from an EMC Isilon or Unity VSA cluster
- Creating a non-administrator user for an EMC Isilon cluster
- Utilizing access zone's SmartConnect Zone/Alias mappings
- Purging the audit logs in an Isilon filer
- Configuring EMC Unity VSA file servers
- Configuring Hitachi NAS file server monitoring
- Configuring Windows File Server monitoring
- Configuring Veritas File System (VxFS) file server monitoring
- Configuring monitoring of a generic device
- Managing file servers
- About configuring filers
- Viewing configured filers
- Adding filers
- Add/Edit NetApp filer options
- Add/Edit NetApp cluster file server options
- Add/Edit EMC Celerra filer options
- Add/Edit EMC Isilon file server options
- Add/Edit EMC Unity VSA file server options
- Add/Edit Windows File Server options
- Add/Edit Veritas File System server options
- Add/Edit a generic storage device options
- Add/Edit Hitachi NAS file server options
- Custom schedule options
- Editing filer configuration
- Deleting filers
- Viewing performance statistics for file servers
- About disabled shares
- Adding shares
- Managing shares
- Editing share configuration
- Deleting shares
- About configuring a DFS target
- Adding a configuration attribute for devices
- Configuring a DFS target
- About the DFS utility
- Running the DFS utility
- Importing DFS mapping
- Renaming storage devices
- Configuring NetApp 7-mode file server monitoring
- Section IV. Configuring SharePoint data sources
- Configuring monitoring of SharePoint web applications
- About SharePoint server monitoring
- Credentials required for configuring SharePoint servers
- Configuring a web application policy
- About the Data Insight web service for SharePoint
- Viewing configured SharePoint data sources
- Adding web applications
- Editing web applications
- Deleting web applications
- Adding site collections
- Managing site collections
- Removing a configured web application
- Configuring monitoring of SharePoint Online accounts
- About SharePoint Online account monitoring
- Configuring user with minimum privileges in Microsoft 365
- Creating an application in the Microsoft Azure portal
- Configuring application without user impersonation for Microsoft 365
- Adding SharePoint Online accounts
- Managing a SharePoint Online account
- Adding site collections to SharePoint Online accounts
- Managing site collections
- Configuring monitoring of SharePoint web applications
- Section V. Configuring cloud data sources
- Configuring monitoring of Box accounts
- Configuring OneDrive account monitoring
- Configuring Azure Netapp Files Device
- Managing cloud sources
- Section VI. Configuring Object Storage Sources
- Section VII. Health and monitoring
- Section VIII. Alerts and policies
- Section IX. Remediation
- Configuring remediation settings
- Section X. Reference
- Appendix A. Data Insight best practices
- Appendix B. Migrating Data Insight components
- Appendix C. Backing up and restoring data
- Appendix D. Data Insight health checks
- About Data Insight health checks
- Services checks
- Deployment details checks
- Generic checks
- Data Insight Management Server checks
- Data Insight Indexer checks
- Data Insight Collector checks
- Data Insight Windows File Server checks
- Data Insight SharePoint checks
- Classification server health checks
- Data Insight self service portal server health checks
- About Data Insight health checks
- Appendix E. Command File Reference
- Appendix F. Data Insight jobs
- Appendix G. Troubleshooting
- About general troubleshooting procedures
- About the Health Audit report
- Location of Data Insight logs
- Downloading Data Insight logs
- Migrating the data directory to a new location
- Troubleshooting FPolicy issues on NetApp devices
- Troubleshooting EMC Celera or VNX configuration issues
- Troubleshooting EMC Isilon configuration issues
- Troubleshooting SharePoint configuration issues
- Troubleshooting Hitachi NAS configuration issues
- Troubleshooting installation of Tesseract software
- Troubleshooting RHEL 9 upgrade issue
Enabling SSL support for Cluster Mode NetApp auditing
Before enabling SSL support for Cluster Mode NetApp auditing, note the supported NetApp External FPolicy Engine Configuration:
Data Insight supports only NetApp's FPolicy 'server-auth' mode for secure communication which means NetApp verifies the identity of the Data Insight FPolicy Server.
The 'mutual-auth' mode is not supported.
The 'no-auth' mode is supported for non-secure communication.
Supported Certificate Types:
Only Self-signed server certificates are supported at the moment.
These enable encryption of data in motion between NetApp and Data Insight but are not signed by a Root Certificate Authority.
Security Protocols
TLS Protocol: Only TLS 1.2 and above is enabled on the Data Insight FPolicy Server.
Cipher Suites: Only ECDHE ciphers are enabled on the &ProductName_generic; FPolicy Server. For example, a NetApp 9.3 cluster in a test environment negotiated the following cipher which is the strongest available in TLS 1.2: ECDHE-RSA-AES256-GCM-SHA38 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD
Supported Collector Operating System
Only Windows collector supports SSL for Cluster Mode NetApp auditing. Linux collector does not support SSL for Cluster Mode NetApp auditing.
Complete the following steps to enable SSL support for Cluster Mode NetApp auditing in Data Insight:
- Create SSL Certificate for each Data Insight FPolicy Server (Data Insight Collector Node) that will receive events from a NetApp SVM
Any suitable tool or PKI solution can be used to generate certificates. Example instructions for using the OpenSSL Toolkit are given below. OpenSSL binaries can be downloaded for use on Microsoft Windows or installed on Linux servers. For more details, visit https://www.openssl.org/community/binaries.html
Create a self-signed certificate for each Data Insight server that will receive FPolicy events from NetApp SVMs. Make a note of the Data Insight Collector Node names that you need to create certificates for. This is normally the Fully Qualified Domain Name (FQDN) of the node but can also be the NetBIOS Hostname if a FQDN is not in use. For example, the FQDN could be something like "DI_collector_1.acme.com".
Run the following OpenSSL command which will ask you to enter the certificate Subject Name fields and then create two files to be used in steps 2 and 3:
>> openssl req -x509 -newkey rsa:4096 -sha256 -nodes -days 1000 -outform PEM -keyout <server name>.key -out <server name>.pem
For example: openssl req -x509 -newkey rsa:4096 -sha256 -nodes -days 1000 -outform PEM -keyout DI_collector_1.acme.com.key -out DI_collector_1.acme.com.pem
Note the following points:
The certificates are valid for 1,000 days but can be changed to fit your certificate expiry policy - simply change the '-days 1000' parameter to increase or decrease the number of days.
Entering values for the Subject Name when prompted by the OpenSSL tool:
The values you enter here do not affect how data is encrypted, but you may have internal policies which say that certificates should have valid Subject Names. Here is an example where the common name is the same as the server name for which the certificate will be used.
Country Name (2 letter code) [AU]: US
State or Province Name (full name) [Some-State]: California
Locality Name (eg, city) []: Santa Clara
Organization Name (eg, company) [Internet Widgits Pty Ltd]: Veritas
Organizational Unit Name (eg, section) []: Engineering
Common Name (e.g. server FQDN or YOUR name) []: DI_collector_1.acme.com
Email Address []: someone@veritas.com
- Install the Certificates on the NetApp SVM(s)
You need to perform this step on each SVM that will be sending audit events to Data Insight Collectors (FPolicy Servers). A Storage Virtual Machine sends audit events to one Data Insight Collector, so you will install the server certificate you created for that Data Insight Collector node into the SVM that is linked to that Data Insight Collector Node. You do not need to install all the certificates generated in Step 1 in all SVMs.
For example, if you have the following two Data Insight Collector Nodes:
Data Insight Collector node '1' is configured to receive events from NetApp SVM 1
Data Insight Collector node '2' is configured to receive events from NetApp SVM 2
You would install the server certificate generated for collector node 1 into SVM 1 and the certificate for collector node 2 into SVM 2 by completing the follow:
SSH to NetApp using cluster management IP/hostname.
Identify or choose the data SVM to which you want to configure in Data Insight for SSL audit monitoring. Command to view the SVM is:
>> vserver showInstall the certificate of the Data Insight Collector Node that this SVM will send audit events to. You will need the contents of the <server>.pem file created in step 1. Use the following command to install certificate on your data SVM:
>> security certificate install -type client-ca -vserver <data SVM name>You will be prompted to paste the certificate. Ensure you paste the entire contents of the <server>.pem file including the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" without any spaces or newlines after "-----END CERTIFICATE-----".
Verify that the certificate has been installed by showing the list of certificates:
>> security certificate show -vserver <data SVM name>
Note:
You do not need to configure the FPolicy's external-engine on SVMs manually - this is done automatically by Data Insight when you configure or reconfigure the FPolicy Service (see the last step below).
- Enable SSL support for Cluster Mode NetApp in Data Insight collector node
Perform the following steps on each Data Insight Collector Node that will receive FPolicy events from a NetApp SVM
Copy the Data Insight FPolicy Server's certificate files ('<server>.pem' and '<server>.key') to the Data Insight Collector Node. For example, you would copy the certificates generated for Data Insight Collector Node '1' to Data Insight Collector Node '1' and not Data Insight Collector Node '2'.
We recommend that they are copied to the Data Insight data directory (e.g. 'C:\DataInsight\data') and that you use the same folder on each node for consistency. But the certificate can be placed in any folder on the collector node. In the next step, you will tell Data Insight which folder they are in, so make a note of it.
- Configure the Data Insight FPolicy Server to support SSL Communication with NetApp SVMs
Perform the following steps only on the Data Insight Management Server. Make sure that you have the ID of each Data Insight collector node and ID of each monitored Cluster Mode NetApp's SVM/filer that you need to configure. The collector node ID and SVM/filer ID are numbers that you can be find in the Data Insight Management Portal UI. To find the IDs:
For collector node id:
Navigate to Settings
Click Data Insight Servers under Inventory
Click top bar and click down arrow
Select columns and check box next to the ID field.
Note down the collector IDs for which you want to configure Cluster Mode NetApp's SSL auditing
For SVM/filer id:
Navigate to Settings
Click Filers under Inventory
Click top bar and click down arrow
Select columns and check box next to the ID field.
Note down the monitored SVM/filer IDs for which you want to configure Cluster Mode NetApp's SSL auditing.
Now, add some new configuration properties to the Data Insight Configuration database. The properties will control the behavior of the FPolicy Service which are:
- fpolicydcmod.ssl
Node level obj attribute. Integer value - set to 1 to enable SSL auditing support for Cluster Mode NetApp.
- fpolicydcmod.ssl.srv.cert
Node level obj attribute. String value - the absolute file path for the FPolicy server's PEM certificate file ('<server>.pem file') The certificate must be in PEM format, the one generated in step 1, above. File extension must be ".pem".
- fpolicydcmod.ssl.srv.cert.key
Node level obj attribute. String value, absolute file path for the SSL server certificate's key file ('<server>.key'). - vserver.sec.adt.conn Device/filer level obj attribute. Integer value, default value is 0, 1 to configure secure auditing communication at filer/device/vserver level. This obj attribute need to be configured for every SVM device/filer for which you need SSL auditing support.
Optional properties:
- fpolicydcmod.ssl.port (optional)
Optional attribute. Node level obj attribute. Integer value. Port number for SSL server, default value is 20249 from backend side. Value can be a valid port number for FPolicy SSL/secured Server.
- fpolicydcmod.ssl.ciphers (optional)
Optional attribute. Node level obj attribute. String value. Default value is "ECDHE". Value can be a valid preferred ciphers string.
On the Data Insight Management Server open a command-line prompt and change to the '<DI install directory>\bin' folder. Run the following commands to add the new configuration properties. You will need to substitute the Data Insight Collector Node ID or Device id for monitored SVM in the --obj-id property and the path to the certificates from each collector node.
>> configdb.exe --add-obj-attribute --type "node" --obj-id <DI collector node id> --attr-name "fpolicydcmod.ssl" --attr-value 1 e.g. >> configdb.exe --add-obj-attribute --type "node" --obj-id 1 --attr-name "fpolicydcmod.ssl" --attr-value 1 >> configdb.exe --add-obj-attribute --type "node" --obj-id <DI collector node id> --attr-name "fpolicydcmod.ssl.srv.cert" --attr-value "<Server certificate file path>" e.g. >> configdb.exe --add-obj-attribute --type "node" --obj-id 1 --attr-name "fpolicydcmod.ssl.srv.cert" --attr-value "C:\DataInsight\data\DI_collector_1.acme.com.pem" >> configdb.exe --add-obj-attribute --type "node" --obj-id <DI collector node id> --attr-name "fpolicydcmod.ssl.srv.cert.key" --attr-value "<Server certificate key file path>" e.g. >> configdb.exe --add-obj-attribute --type "node" --obj-id 1 --attr-name "fpolicydcmod.ssl.srv.cert.key" --attr-value "C:\DataInsight\data\DI_collector_1.acme.com.key" >> configdb.exe --add-obj-attribute --type "filer" --obj-id <Device id for monitored SVM> --attr-name "vserver.sec.adt.conn" --attr-value 1 e.g. >> configdb.exe --add-obj-attribute --type "filer" --obj-id 2 --attr-name "vserver.sec.adt.conn" --attr-value 1 - Note: Repeat this command for every configured SVM device/filer for which you need to enable SSL auditing support. -> optional commands: >> configdb.exe --add-obj-attribute --type "node" --obj-id <DI collector node id> --attr-name "fpolicydcmod.ssl.port" --attr-value <Valid port number> e.g. >> configdb.exe --add-obj-attribute --type "node" --obj-id 1 --attr-name "fpolicydcmod.ssl.port" --attr-value 8788 >> configdb.exe --add-obj-attribute --type "node" --obj-id <DI collector node id> --attr-name "fpolicydcmod.ssl.ciphers" --attr-value "<Valid preferred ciphers string>" e.g. >> configdb.exe --add-obj-attribute --type "node" --obj-id 1 --attr-name "fpolicydcmod.ssl.ciphers" --attr-value "ECDHE"
- Restart the Data Insight FPolicy services and validate events are received
In the last step, you will reconfigure the Data Insight FPolicy Service which will then be able to receive events from the NetApp SVMs over SSL/TLS.
Login to the Data Insight Management Server Portal UI
Configure/start "DataInsightFPolicyCMod" service from the collector node's services page
Note:
This step is necessary to automatically configure the monitored NetApp SVM external-engine configuration to use 'server-auth' (SSL/TLS).
This step is necessary to automatically configure the monitored NetApp SVM external-engine configuration to use 'server-auth' (SSL/TLS).
You can now perform audit events in the monitored shares and validate that events are processed by Data Insight. You can do that by multiple ways like
wait for at least 2 minutes OR restart the 'DataInsightFPolicyCMod' service from Windows services panel on Data Insight collector node machine. You can also check for the presence of temporary audit output files on the Data Insight Collector Node folder '<DI data dir>\collector'. For cluster mode NetApp the filenames start with "fpolicy_".
Wait for the default scheduled Jobs to execute which will ingest the audit events. You can also manually run the Data Insight ingest jobs in the following sequence which will speed up the process:
CollectorJob - Run on Collector Node on which the monitored share is configured.
FileTransferJob - Run on Collector Node on which the monitored share is configured.
IndexWriterJob - Run on Indexer Node on which the monitored share is configured.
Jobs can be found and run from:
Settings -> Data Insight Servers >> select the Collector node from list >> Jobs >> Select the required Job >> Select action >> Run
Once the jobs are successful, you can view audit events in Data Insight Workspace by going to Workspace >> Data Sources >> select and expand filer >> select share name >> Expand Profile >> Audit logs >> configure different filter values and clicking GO.