Seeding FileCloud for Amazon S3

To use the seeding tool, MongoDB should be enabled and running in PHP CLI mode.

• This may require you to edit the PHP.ini file.
• In Windows, the MongoDB module is already enabled by default.
• If MongoDB is not enabled for PHP CLI mode, the tool will fail.

Enable MongoDB:

In Linux enter:

[root@cnfc php.d]# php -m | grep mongodb
mongodb
[root@cnfc php.d]#

In Windows enter:

C:\Windows\system32>C:\xampp\php\php.exe -m | findstr mongodb
mongodb

C:\Windows\system32>

If you do not get the above results, please Contact FileCloud Support.

1. In a command line enter:
   For Windows:

   CODE

   cd c:\xampp\htdocs\resources\tools\seeding
   PATH=%PATH%;C:\xampp\php

   For Linux:

   CODE

   cd /var/www/html/resources/tools/seeding

   Then, for both Windows and Linux, enter:
   

   CODE

   php  seed.php -h <hostname> -p <from path> -i -d <storagepath> -m <s3inipath> -u <user> -r --trim-dot

   Parameters:
   [required] -h <host> Site host name or 'default' for default site.
   [required] -p <from path> Source path from which files are seeded.
   [optional] -i   Seed files. Set this flag to seed files.
   [optional] -d  <storagepath> Seed files from source path specified with -p to this existing storage path.
   [optional] -m <s3inipath> Path to migrate.ini. This ini file will be used to migrate existing local storage to S3 storage.
   [optional] -u  <user> User whose files are to be imported. File with the user name should exist in the source path. Applicable only with -i.
   [optional] -r   Reset existing database. This will reset the databases, so use it with caution.
   [optional] --trim-dot   For right-to-left languages, removes periods the tool perceives as being in the wrong position and therefore invalid characters. --trim-dot is a flag; it does not have a value.

The seed command deletes the source files after uploading. This is designed to improve the speed of seeding.

To seed files for multiple users at the same time, prepare a top directory(source path) with separate folders for each user to be seeded with data. Under the user specific folder place files/folders to be seeded for that user.
Run the following command to seed all the users from the prepared source path.

php seed.php -h default -p /tmp/seedfolder -i

The following code shows how to export files for user 'jdoe' from site site21.hostedcloud.com to directory 'cloudexport'.

php seed.php -h default -p C:\temp\seedfolder -i

Note

• If user account exists, seeded files/folders will be imported to those accounts
• If user accounts doesn't exist, user accounts will be created before seeding.

To seed files for multiple users at the same time, prepare a top directory(source path) with separate folders for each user to be seeded with data. Under the user specific folder place files/folders to be seeded for that user.
Run the following command to delete all the existing data and seed from the prepared source path.

php seed.php -h default -p /tmp/seedfolder -i -r

php seed.php -h default -p C:\temp\seedfolder -i -r

Note

• All the existing user accounts and its associated data will be deleted before the seeding.
• New user accounts will be created before seeding. Default username and password will be used (i.e password → password)

To seed files into an existing FileCloud storage path, prepare a top directory(source path) with a single folder under which files/folders to be seeded are placed. 
Run the following command to seed the single folder and its contents to an existing FileCloud storage path.

php seed.php -h default -p /tmp/seedfolder -d /jdoe/march

php seed.php -h default -p C:\temp\seedfolder -d /jdoe/march

Note

• In this case, the command imports a single folder under the source path into the FileCloud storage path /jdoe/march. All contents of seedfolder are copied to /jdoe/march, but the folder seedfolder is not copied.

The seeding tool can also migrate files from local storage to S3 storage. When the tool is run in this mode, it does the following steps:

1. Checks if AWS CLI is installed on the system running the tool
2. Checks if a valid migration ini file is specified. Look below for the file format.
3. Important: Deletes the existing S3 storage database. If the site was never configured for S3 before, then this should not be an issue.
4. Creates a new S3 storage database and imports the data from local storage database converting it into S3 storage database format on the fly.
5. Creates multiple AWS CLI commands to upload data from the local storage to the S3 bucket. The details of this transfer are generated using the specified ini file.
6. Executes the AWS CLI commands prepared in the previous step.

Run the following command to migrate from local storage to S3 storage

php seed.php -h default -m /tmp/migrates3.ini

php seed.php -h default -m C:\temp\migrates3.ini

aws_storage_bucket = "company.bucket"
aws_storage_folderprefix = "site1"
aws_access_key_id = "AKIAT4YDRDUSRO863KJJ"
aws_secret_access_key = "stPwbS3Y1KrZGUkVbNcYJx+8S/ZZKFROOjUdG9e9"
aws_region = "us-east-1"

If you are migrating to FileCloud Online, the full set of databases has to be exported along with migration. This can be achieved with the following commands.
Linux migrating from local storage to S3 storage

Windows migrating from local storage to S3 storage

Contact FileCloud Support for help with this procedure.