Today is one of those remarkable days in the history of our Company for two reasons. First, we’ve released a new plugin, the absence of which stopped a number of prospects from purchasing our classified ads software. Second, the new add-on is number 90 on our plugin list. Actually, we put the add-on, which would allow you to store media files in cloud storage, on our to-do list a few years ago. The day has come! Meet our new add-on called Remote Storage. The new Plugin allows you to offload the main server by transferring and storing photo and video files of listings and user avatars in cloud storage.
For a better understanding of how the new plugin Remote Storage works, it would be useful to recall how images are handled in the basic software version.
Once a user adds his listing, the script will save the original photo on the server for further manipulation with it. In particular, the script will create a few copies of the image of different sizes, which includes thumbnails for desktops and smartphones and a preview image for the gallery on the Listing Details page.
Depending on the settings of membership plans and listing packages on your site, a listing might contain 5-10 and even more photos. Now multiply the number of images by the number of created copies and medium file size and the average multimedia stack will add up to 500 Kb to a few megabytes for a listing excluding video files, which might be uploaded by users to their listings.
The more listings you have on your site the more disk space you will need for storing images and video files. For increasing the disk space you will have to upgrade to a more expensive hosting plan, which is not always a good idea because with boosting the disk space you get excessive hosting parameters that you never planned to purchase.
That’s when storing media files on Amazon S3-compatible cloud storage comes in handy. Storing media files in cloud storage is remarkable for,
- Simplicity of use
- Unlimited disk space;
- Relatively inexpensive cloud storage solutions;
- A number of offers on the market.
The fact that all cloud storage providers bill for traffic between customers and media files in the cloud makes storing files in the cloud more attractive. It allows you to keep the hosting expenses at an affordable level.
How the Remote Storage plugin works
Your users won’t see how the Remote Storage works in the user interface except for image load speed. As practice shows, if you select a cloud storage data center that will be located close to your customers the load speed of media files may even rise. As a whole, all manipulations with media files take place under the hood of the Plugin.
To make the Plugin work, you will need to select a cloud storage service that you want to use and get two keys – access key ID and secret access Key. The majority of cloud storage companies work on a freemium basis. That is, you create an account and add your card to the account and start using the service until you exceed the limit. You will be billed once you exceed the limit.
After getting the keys, you will need to proceed to the Flynax admin panel to the Plugins section. Install the Remote Storage plugin and create a bucket to which your media files will be transferred. Then you add a bucket name, select a region for the bucket as we wrote before, select a server that is located close to your customers for faster loading of media files, enter the keys you got earlier, and click Add. That’s it, the bucket is ready to accept the files.
Once a user added his listing, the Plugin will move photos and video files to cloud storage and rewrite paths to files. Our classified ads script generates the following type of links to images,
http://domain.com/files/03-2023/ad132/rerum-dolor-consequat-sapiente-amet-nostrud-volu-2033771382_large.jpg
Once the plugin moves images to the cloud storage, it will generate links of the following type,
https://s3.eu-west-2.amazonaws.com/domain.com-6412f5f656258/listings/03-2023/ad132/rerum-dolor-consequat-sapiente-amet-nostrud-volu-2033771382_large.jpg
This way, the Plugin will rewrite paths to photos and videos and when browsed next time, the listing will be loading media files from a remote server, rather than from the main server where the software is installed. The Plugin may handle only images and video files uploaded to listings and user thumbnails as well.
We tried to protect you from data loss as much as possible because losing media files is a critical issue for a listing. So when removing a bucket, the Plugin will suggest moving all the available files back to the main server. After transferring the files from the bucket back to the server, Pugin will remove the bucket from the cloud storage. For this reason, we recommend against managing the bucket directly from the cloud storage console (modifying files, changing bucket permission, etc) because that might result in interruption of the connection and consequently data loss.
Shall I make a back-up before moving to the cloud
If your hosting company allows you to make a back-up of the files then we strongly recommend that you back up if you run your site for some time and want to move media files of already added listings. We recommend making a backup before moving the files to a remote server and before moving them back to the main server. There are a few factors that might compromise the security of the transfer,
- Connection failure during moving files;
- Unforeseen circumstances on the server;
- Hacker attack.
Alternative buckets
The Plugin allows you to create one or a few alternative buckets in case the main bucket fails to accept the files. The manager also allows you to track the number of files contained in every bucket.
When you might need the Remote Storage
There are two scenarios when using the Remote Storage plugin makes sense,
- Your database is actively growing and you’ve almost run out of disk space;
- You decided to move and store the media files in the cloud storage in order to offload the main server.
What you need to keep in mind when moving to cloud storage
When running the Plugin you should fully understand,
- Pricing policy for storing files on a cloud storage service;
- Files will be stored on a third-party server. No copies will be made and stored on your web server;
- Removal of a bucket in the cloud storage console will result in losing all your files irreversibly;
- Inability to pay for the bucket may result in losing all your files;
- Uninstalling the Plugin without transferring the files to your web server will result in losing all your files too;
- Before transferring numerous media files of your site to your web server, make sure it offers enough space.
What cloud providers are supported by the Plugin
Of course, you may find a lot of cloud solutions on the Internet today but we integrated the 5 most popular cloud providers at this stage, which is more than enough for version 1.0.
- Amazon S3
- DigitalOcean Spaces
- Wasabi Hot Cloud Storage
- VK Cloud Storage
- Yandex Object Storage
We believe that the Remote Storage plugin will become a good help in file optimization on your classified ads site. It will allow you to offload the main server and possibly speed up the loading of media files, as well as avoid unwanted costs for hosting services. The Plugin is very easy to set up. Once configured, the Plugin will work smoothly in automatic mode, invisibly doing its important work for you. Even if you change your mind about using the Remote Storage plugin at some point, you may easily transfer the files to the main server. The Remote Storage plugin is a great help to a big project.