@InterfaceAudience.Public @InterfaceStability.Evolving public class DistCp extends Configured implements Tool
Constructor and Description |
---|
DistCp(Configuration configuration,
DistCpOptions inputOptions)
Public Constructor.
|
Modifier and Type | Method and Description |
---|---|
protected void |
cleanup()
Clean the staging folder created by distcp.
|
Job |
createAndSubmitJob()
Create and submit the mapreduce job.
|
protected Path |
createInputFileListing(Job job)
Create input listing by invoking an appropriate copy listing
implementation.
|
Job |
execute()
Original entrypoint of a distcp job.
|
Job |
execute(boolean extraContextChecks)
Implements the core-execution.
|
protected org.apache.hadoop.tools.DistCpContext |
getContext()
Returns the context.
|
protected Path |
getFileListingPath()
Get default name of the copy listing file.
|
static void |
main(String[] argv)
Main function of the DistCp program.
|
int |
run(String[] argv)
Implementation of Tool::run().
|
void |
waitForJobCompletion(Job job)
Wait for the given job to complete.
|
getConf, setConf
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getConf, setConf
public DistCp(Configuration configuration, DistCpOptions inputOptions) throws Exception
configuration
- configuration against which the Copy-mapper must runinputOptions
- Immutable optionsException
public int run(String[] argv)
public Job execute() throws Exception
execute(boolean)
without doing extra context checks and setting some configs.Exception
- when fails to submit distcp job or distcp job failspublic Job execute(boolean extraContextChecks) throws Exception
extraContextChecks
- if true, does extra context checks and sets some configs.Exception
- when fails to submit distcp job or distcp job fails, or context checks failpublic Job createAndSubmitJob() throws Exception
Exception
public void waitForJobCompletion(Job job) throws Exception
job
- the given mapreduce job that has already been submittedException
protected Path createInputFileListing(Job job) throws IOException
job
- - Handle to jobIOException
- - If anyprotected Path getFileListingPath() throws IOException
IOException
- - Exception if anyprotected org.apache.hadoop.tools.DistCpContext getContext()
public static void main(String[] argv)
argv
- Command-line arguments sent to DistCp.protected void cleanup()
Copyright © 2024 Apache Software Foundation. All rights reserved.