Read HBase release-0.8.1

[Hilmi Yildirim]

Hi,
I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed
my hbase-site.xml with the correct configurations into the resource
folder. When I start the job locally then the job uses the config inside
the hbase-site.xml and it works great. When I want to run the job on a
cluster then the job uses the default hbase config instead of the config
defined in the hbase-site.xml. The hbase-site.xml is included in the fat
jar. To start the job I use the web interace.

Best Regards,

--
--
Hilmi Yildirim
Software Developer R&D

T: [phone]
[hidden email]

http://www.neofonie.de

Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/

Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh

Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko

[Stephan Ewen]

I am not an HBase expert. How does HBase find the hbase-site.xml? Does it simply search through the classpath? 

If it uses the classpath, can you see what hbase-site.xml is used?

  - In your user code, you can do a getClass().getClassloader().gerResource("hbase-site.xml") and see where it is?

  - Can you do the same with ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and see if that returns something different?

Thanks,

Stephan

On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim <[hidden email]> wrote:

Hi,
I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed my hbase-site.xml with the correct configurations into the resource folder. When I start the job locally then the job uses the config inside the hbase-site.xml and it works great. When I want to run the job on a cluster then the job uses the default hbase config instead of the config defined in the hbase-site.xml. The hbase-site.xml is included in the fat jar. To start the job I use the web interace.

Best Regards,

--
--
Hilmi Yildirim
Software Developer R&D

T: <a href="tel:%2B49%2030%2024627-281" value="[phone]" target="_blank">[phone]
[hidden email]

http://www.neofonie.de

Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/

Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh

Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko

[Hilmi Yildirim]

When I execute both job locally then the result of both methods is the same. But when I execute the job on the cluster then the first method returns a path to the hbase-site.xml but the second method returns null.

Best Regards,
Hilmi

Am 28.05.2015 um 10:51 schrieb Stephan Ewen:

I am not an HBase expert. How does HBase find the hbase-site.xml? Does it simply search through the classpath? 

If it uses the classpath, can you see what hbase-site.xml is used?

  - In your user code, you can do a getClass().getClassloader().gerResource("hbase-site.xml") and see where it is?

  - Can you do the same with ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and see if that returns something different?

Thanks,

Stephan

On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim <[hidden email]> wrote:

Hi,
I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed my hbase-site.xml with the correct configurations into the resource folder. When I start the job locally then the job uses the config inside the hbase-site.xml and it works great. When I want to run the job on a cluster then the job uses the default hbase config instead of the config defined in the hbase-site.xml. The hbase-site.xml is included in the fat jar. To start the job I use the web interace.

Best Regards,

--
--
Hilmi Yildirim
Software Developer R&D

T: <a moz-do-not-send="true" href="tel:%2B49%2030%2024627-281" value="[phone]" target="_blank">[phone]
[hidden email]

http://www.neofonie.de

Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/

Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh

Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko

[Hilmi Yildirim]

I solved the problem by overwriting the method "configure". There, I used the "addResource" method of the HBaseConfiguration to add the correct hbase-site.xml

Best Regards,
Hilmi

Am 28.05.2015 um 11:21 schrieb Hilmi Yildirim:

When I execute both job locally then the result of both methods is the same. But when I execute the job on the cluster then the first method returns a path to the hbase-site.xml but the second method returns null.

Best Regards,
Hilmi

Am 28.05.2015 um 10:51 schrieb Stephan Ewen:

I am not an HBase expert. How does HBase find the hbase-site.xml? Does it simply search through the classpath? 

If it uses the classpath, can you see what hbase-site.xml is used?

  - In your user code, you can do a getClass().getClassloader().gerResource("hbase-site.xml") and see where it is?

  - Can you do the same with ClassLoader.getSystemClassLoader().getResource("hbase-site.xml") and see if that returns something different?

Thanks,

Stephan

On Thu, May 28, 2015 at 10:14 AM, Hilmi Yildirim <[hidden email]> wrote:

Hi,
I used release-0.8.1for Hadoop 2 to read from HBase. For that I placed my hbase-site.xml with the correct configurations into the resource folder. When I start the job locally then the job uses the config inside the hbase-site.xml and it works great. When I want to run the job on a cluster then the job uses the default hbase config instead of the config defined in the hbase-site.xml. The hbase-site.xml is included in the fat jar. To start the job I use the web interace.

Best Regards,

--
--
Hilmi Yildirim
Software Developer R&D

T: <a moz-do-not-send="true" href="tel:%2B49%2030%2024627-281" value="[phone]" target="_blank">[phone]
[hidden email]

http://www.neofonie.de

Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/

Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh

Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko