Class IPAPassRemoveUnnecessaryCheckpoints

  • public class IPAPassRemoveUnnecessaryCheckpoints
    extends IPAPass
    This rewrite identifies and removes unnecessary checkpoints, i.e., persisting of Spark RDDs into a given storage level. For example, in chains such as pread-checkpoint-append-checkpoint, the first checkpoint is not used and creates unnecessary memory pressure.
    • Constructor Detail

      • IPAPassRemoveUnnecessaryCheckpoints

        public IPAPassRemoveUnnecessaryCheckpoints()
    • Method Detail

      • isApplicable

        public boolean isApplicable​(FunctionCallGraph fgraph)
        Description copied from class: IPAPass
        Indicates if an IPA pass is applicable for the current configuration such as global flags or the chosen execution mode (e.g., HYBRID).
        Specified by:
        isApplicable in class IPAPass
        fgraph - function call graph
        true if applicable.
      • rewriteProgram

        public boolean rewriteProgram​(DMLProgram prog,
                                      FunctionCallGraph fgraph,
                                      FunctionCallSizeInfo fcallSizes)
        Description copied from class: IPAPass
        Rewrites the given program or its functions in place, with access to the read-only function call graph.
        Specified by:
        rewriteProgram in class IPAPass
        prog - dml program
        fgraph - function call graph
        fcallSizes - function call size infos
        true if function call graph should be rebuild
      • rRemoveCheckpointReadWrite

        public static void rRemoveCheckpointReadWrite​(Hop hop)