I am looking for a software program that will run on a cloud Windows Server that periodically scans a dedicated input folder for text files, parses and validates the files for field data, adds the data to a database, and moves the files to a dedicate archive folder. The input files will be from a secure FTP transfer.
GEN 1
• Database – the database shall be a MySQL database. Recommendations are welcome.
• Language – the software language shall be written on the .NET framework in either C# or VB.NET.
• GUI - a GUI shall view file statistics, process activities, errors, and dashboard data.
• Performance – the program shall be capable of processing 1,000 files per minute.
GEN 2
• Error detection – the program shall check for duplicate files, data, formats, etc.
• Compression – the program shall compress archive files.
Developer – It is preferred that the developer has the following capabilities:
• Server Setup – setting up a Microsoft Azure Windows Server
• FTP Server – setting up a secure FTP server
• Software Repository – manage software in a SVN database
• Experience – vast knowledge and experience with similar systems
Hello
I am a full time freelancer having 7.5~ years expertise in MS Technologies(C#). Please visit my profile to know about me and my work.
Well regarding you project, I have gone through description and I can create this application for you.
I will be using C# Win Forms(or a windows service) to create application which will pick files from ftp folder and then parse them and will save to database. After that will archived the files. A GUI will be there to provide statistics of processing. Logging mechanism will also be there to log each processing step based on severity to identify issues(if any).
I prefer using Bitbucker(Mercurial) for Code repository. Also I am well aware of configuration of system end to end viz. setting Cloud WInodws Server, FTP Server, Databaase setup etc.
Hello,
I am VB, VBA, C#, WPF, Java, Delphi, PHP, ASP.NET, CAD and GIS developer, I developed many desktop applications and custom tools, macros and Add-Ins. For reference see my profile.
I am ready to discuss and doing the project.
Regards,
Eyad
My name is Mike and I’m from UK. I work with individual clients and also provide outsourcing services for a number of UK and USA based agencies. Your project description sounds interesting to me and I do have skills & experience that is required to complete this project. I can show you some examples of my work. Please contact me to discuss your project.
Hey there.. Thanks for the details description... Yes, we can go with mySql database (since it is free of cost)......Please reply... We can discuss further and hopefully get it started soon... Thank you...
Proposed solution
Deploment of a SFTP server onto a Azure VM
Provision Azure Blob Storage
Point SFTP Server at Azure Blob Storage
Creation of a data model to store the data from the uploaded files
Creation of a database based on the data model
Creation of code to parse data from files and upload to database.
Creation of duplicate and error handling code that moves problems files to a bad queue.
Creation of Azure Web Job which will use a trigger to fire as soon as a file is uploaded via SFTP (doesn't need to run on a schedule)
Creation of an Azure Web Site that will read from the database and display activity and errors etc
Questions
Are all the files the same format?
Do the files need to be processed in the order they arrive? If a problem file is found do we exclude it and keep processing and pause until its fixed?
Once a problem file is found how do we it? Will it be fixed externally and uploaded again?
Are the files uploaded from the same client or different clients?
Is SFTP definitely required? If you control the upload process then this could be changed to upload directly to Azure Blob storage using Shared Access Signatures. This would negate the need for an SFTP server and lower your hosting costs.
Does a database model exist for these files or does it need to be created?
What else do you plan to do with this data once its in the database? This will drive the database decision and affect your hosting costs Options are MS SQL (expensive), MySQL and Azure Table Storage
Dear
Thank you for your posting.
I'm a PHP developer and I'm very interested in your job post.
I've more than 5 years PHP development Experience.
And I am available to chat by email.
I hope work with you.
Hire me please.
Regards.
Hi:
I have 20 years experienced with Vb, Databases and Windows Servers.
I have done programs to download from FTP server, and I have a Windows Azure subscription.
About you project:
1.- Database: If you have a Windows Server Azure, why not using SQL Server ? it could be the free version (Express). Otherwise, I don't have problems in working with MySQL.
2.- Language: I use Vb.net
3.- Performance: the number of files to process depends on the size of them, FTP transfer speed, the database that will receive the data and other factors. Could be 10.000, more or may be less.
4.- To avoid files duplication, we can store the names of files processed.
5.- The security on FTP is set using login credentials.
6.- We can create a Windows Plan to execute the program periodically.
I am available to start this project. If you have any question i am on freelancer.com chat
Greetings