c# - Running external processes asynchronously in a windows service -

i writing program moves csv files "queue" folder "processing" folder, third party process called import.exe launched taking csv file path argument. import.exe long running task.

i need program continue running , checking queue new files. reason i’ve chosen windows service application long running.

my problem overwhelmed options , can't understand if should approach problem background threads or parallel programming, or combination of both.

so far have code running synchronously. see @ moment, wildly firing processes without anyway manage or check completion. have commented out process.waitforexit () blocking call.

public int maxconcurrentprocesses = 10; protected override void onstart(string[] args)     {         // set timer trigger every minute.         system.timers.timer timer = new system.timers.timer(60000);                     timer.elapsed += new system.timers.elapsedeventhandler(this.ontimer);         timer.start();     }  private void ontimer(object sender, system.timers.elapsedeventargs args)     {                     // how many instances of import.exe running?         process[] importprocesses = process.getprocessesbyname("import");         int countrunning = importprocesses.count();          // if there less maxconcurrentprocesses, create many needed reach maxconcurrentprocesses         if (countrunning < maxconcurrentprocesses)         {             int processestostart = maxconcurrentprocesses - countrunning;             (int = 0; < processestostart; i++)             {                 fireoffimport();             }         }     }  private void fireoffimport()     {         // first file returned queue folder         string filepathsource = getnextcsvinqueue();          if (filepathsource != "")         {             // …             // commandarguments = create our arguments here             // …                     // move file processing folder here             // …               // give new process import tool location , arguments             processstartinfo startinfo = new processstartinfo(importlocation + "\\import.exe", commandarguments);             try             {                 process process = process.start(startinfo);                 // process.waitforexit(20000);                 // if process has exited, there 4 csv files created in same directory file.                               }             catch (exception ex)             {                // deal exception here             }        }     } 

i tried creating array of tasks, , running these asynchronously. @ end still had call task.waitall() before read result. if 1 finishes early, must wait longest running task.

i think need try loop through creating processes asynchronously perhaps using tasks, don’t understand how background process, can keep service timer checking number of processes if needs create more.

the first improvement of code coming mind remove timer , replace system.io.filesystemwatcher , event handler created event. way, code needs not administrate files in queue before , new ones have arrived. less code = less problems, usually.

second, taking word "task" hints performing complete import task in 1 system.io.tasks.task instance, including spawning corresponding import process instance , waiting exit when done.

if wish limit amount of import processes running @ time, alternative of bookkeeping code does, replace scheduler scheduler limits amount of tasks, allowed run in parallel compared default scheduler. if each task associated 1 importer instance , maximum n tasks allowed run concurrently, have maximum n instances of importer process.

the code below shows (in form of console application), how above described like, minus custom scheduler covered in provided link.

using system.threading.tasks;  namespace consoleapplication4 {     class program     {         static string importerprocessname = "import.exe";         static string rootfolder = @"e:\temp\a\";         static string queuepath = system.io.path.combine(rootfolder, "queue" );         static string processingpath = system.io.path.combine(rootfolder, "processing");         static string donepath = system.io.path.combine(rootfolder, "done");         static void main(string[] args)         {             grantfolders(); // make sure have our folders ready action...             var watcher = new system.io.filesystemwatcher(queuepath, "*.txt");             watcher.created += watcher_created;             watcher.enableraisingevents = true;             system.console.readline();         }         static task processfile( string filename )         {             task task = new task(() =>             {                 system.console.writeline("processing: " + filename);                 system.io.file.move(system.io.path.combine(queuepath, filename), system.io.path.combine(processingpath, filename));                 string commandline = "-import " + system.io.path.combine(processingpath, filename);                 using (var importer = new system.diagnostics.process())                 {                     importer.startinfo = new system.diagnostics.processstartinfo(importerprocessname, commandline);                     importer.start();                     importer.waitforexit(20000);                     system.io.file.move(system.io.path.combine(processingpath, filename), system.io.path.combine(donepath, filename));                     system.console.writeline("done with: " + filename);                 }             });             return task;         }         static void watcher_created(object sender, system.io.filesystemeventargs e)         {             system.console.writeline("found in queue: " + e.name);             var task = processfile(e.name);             task.start();         }          private static void grantfolders()         {             string[] paths = new string[] { queuepath, processingpath, donepath };             foreach( var path in paths)             {                 if(!system.io.directory.exists(path))                 {                     system.io.directory.createdirectory(path);                 }             }         }     } }