[GPGPU] Synchronize after each kernel, not each copy out

[grosser]

Summary

This change reduces the overall number of synchronize calls for kernels with
a lot of output data at the cost of additional synchronize calls for kernels
launched in sequence without any device to host transfers in between. As the
latter pattern is a lot less frequent, this seems a better tradeoff.

Even though the above motivation would be motivation enough, this is just
a step towards enabling ppcg to not compute to and from device copy calls
at all, which would be incorrect in case we still relied on these calls to
place our synchronization statements.

[bollu]

Does this patch depend on some other patch? If so, please record this information by creating a Parent Revision. (Edit Related Revisions -> Edit Parent Revision). If some other patch depends on this, then make this a parent of that patch. Helps when looking up revisions later on.

Other than that, LGTM.

lib/CodeGen/PPCGCodeGeneration.cpp

void GPUNodeBuilder::createUser(__isl_take isl_ast_node *UserStmt) {
   isl_ast_expr *StmtExpr = isl_ast_expr_get_op_arg(Expr, 0);
   isl_id *Id = isl_ast_expr_get_id(StmtExpr);
   isl_id_free(Id);
   isl_ast_expr_free(StmtExpr);
 
   const char *Str = isl_id_get_name(Id);
   if (!strcmp(Str, "kernel")) {
     createKernel(UserStmt);
+    if (PollyManagedMemory)
+      createCallSynchronizeDevice();
     isl_ast_expr_free(Expr);
     return;
   }
   if (!strcmp(Str, "init_device")) {
     initializeAfterRTH();
     isl_ast_node_free(UserStmt);
     isl_ast_expr_free(Expr);
     return;
if (isPrefix(Str, "to_device")) {
     isl_ast_expr_free(Expr);
     return;
   }
 
   if (isPrefix(Str, "from_device")) {
     if (!PollyManagedMemory) {
       createDataTransfer(UserStmt, DEVICE_TO_HOST);
     } else {
-      createCallSynchronizeDevice();
       isl_ast_node_free(UserStmt);
     }
     isl_ast_expr_free(Expr);
     return;
   }
 
   isl_id *Anno = isl_ast_node_get_annotation(UserStmt);
   struct ppcg_kernel_stmt *KernelStmt =