天道酬勤,学无止境

rJava fails while invoking RJDBC library from OpenCPU API

Goal : To use RJDBC with OpenCPU AJAX Post call.

Use case : I need to connect with Apache phoenix database and get some data. And on that data I am supposed to do analytics. So to connect with Phoenix i am using rjdbc(phoenix doesn't support any other option to connect).

Environment : public.opencpu.org

Here is what I am trying:

html

<textarea>
library(RJDBC)
.. do some data fetching and then r statistics
</textarea>
<br />
<button id="submitbutton" >Execute</button>
<br />
<br />
<div id="output">
</div>

javascript

<script type="text/javascript">
  //When Document is Ready
    $(function () {
        var baseurl = "https://public.opencpu.org";
        //because identity is in base
        ocpu.seturl(baseurl+"/ocpu/library/base/R");
        //actual handler
        $("#submitbutton").on("click", function(){
            //arguments
            var mysnippet = new ocpu.Snippet($("#input").val());
            //perform the request
            var req = ocpu.call("identity", {
                "x" : mysnippet
            }, function(session){
                session.getStdout(function(outtxt){
                        $("#output-txt").text(outtxt);   
                });
            });   
            //if R returns an error, alert the error message
            req.fail(function(){
                alert("Server error: " + req.responseText);
                $("#output").hide();
            });      
            req.always(function(){
                $("button").removeAttr("disabled");  
            });
        });    
    });

</script>

scenario

when i click execute button, gets the user entered value from text area ( here library(RJDBC) and some db operations) and sends a ajax post and responds back with the below error

google chrome console log :

POST https://public.opencpu.org/ocpu/library/base/R/identity 400 (Bad Request) 
OpenCPU error HTTP 400
package 'rJava' could not be loaded

Hope this gives better clarity.

Need to confirm whether rJava works with OpenCPU, if so any possible work around to fix this?

标签

评论

The core of my solution looks like the below. The code is taken from an R6 class that has some logging. For the sake of documenting what is happening I've left in the private$log commands, but if running a version of this code you'd want to leave it out. Similarly, there are calls there to functions that you don't really need to get the job done and I pack up the host information in a list rather than pass individual variables for the passwords etc etc. In short, the code I provide here is not minimally replicable.

However, the code I provide at this gist ought to be somewhat replicable (once you provide the database connection, the classPath and driverClass). That gist has been subject to about 0 clean-up, so there is a ton of code in there that is strictly speaking superfluous to getting the job done. None of this code is pretty, elegant, etc - generally I'm ashamed of it, but in the spirit of sharing, I put it up anyway.

Explanatory code segment:

        jvmNode <- makeCluster(1, methods=FALSE, port = sample(11000:11999,1), rscript_args="--vanilla")
          connected <- isTRUE(try(clusterEvalQ(jvmNode, 2)==2, silent=TRUE))
        private$log$finest("forkable node started")
        clusterEvalQ(jvmNode,{library(DBI);library(rJava);library(RJDBC)})

        clusterEvalQ(jvmNode, systemRAMFree <- function() {
  #in GB, platform dependent 
  as.numeric(system('FREE_KB=$(($(echo `sed -n \'2p;3p;4p\' <  /proc/meminfo | sed "s/ \\+/ /g" | cut -d\' \' -f 2 ` | sed "s/ /+/g")));echo $FREE_KB', intern=TRUE))/1024/1024
})       
        clusterEvalQ(jvmNode, options(java.parameters = paste0("-Xmx",systemRAMFree(),"g")))

        hostInformation <- private$hostInformation
        classPath <- "~/redshift/redshiftJDBC41.jar"
        driverClass <- private$driverClass
        java.parameters <- getOption("java.parameters")
        toExportList <- list(statement,driverClass, classPath, hostInformation, java.parameters)
        names(toExportList) <- c("statement","driverClass","classPath", "hostInformation", "java.parameters")
        toExport <- as.environment(toExportList)
        clusterExport(jvmNode, ls(toExport), envir=toExport)
        clusterEvalQ(jvmNode, options(java.parameters = java.parameters))

        clusterEvalQ(jvmNode, rJava::.jinit())
        private$log$finest("JVM Started")
        clusterEvalQ(jvmNode, drv <- RJDBC::JDBC(driverClass, classPath))
        private$log$finest("Driver obtained")

        clusterEvalQ(jvmNode, conn <- dbConnect(drv, 
                                       url = hostInformation$host, 
                                       user = hostInformation$user, 
                                       password = hostInformation$password))
        private$log$debug(logPrefix("query:", statement))
        result <- try(clusterEvalQ(jvmNode, dbGetQuery(conn, statement))[[1]])
        clusterEvalQ(jvmNode, dbDisconnect(conn))
        stopCluster(jvmNode)

受限制的 HTML

  • 允许的HTML标签:<a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • 自动断行和分段。
  • 网页和电子邮件地址自动转换为链接。

相关推荐
  • R 和 Hive 之间的连接(Connectivity between R and Hive)
    问题 我正在尝试在 RStudio(在我的机器上)和 Hive(在不同的服务器上设置)之间建立连接。 这是我的 R 代码: install.packages("RJDBC",dep=TRUE) require(RJDBC) drv <- JDBC(driverClass = "org.apache.hive.jdbc.HiveDriver", classPath = list.files("C:/Users/37/Downloads/hive-jdbc-0.10.0.jar", pattern="jar$",full.names=T), identifier.quote="'") 这是我在执行上述命令时遇到的错误: .jfindClass(as.character(driverClass)1) 中的错误:找不到类 conn <- dbConnect(drv, "jdbc:hive2://65.11.23.453:10000/default", "admin", "admin") 我从这里下载了 jar 文件并将它们放在CLASSPATH 中。 请告知我是否做错了什么以及我如何才能使其正常工作。 谢谢。 回答1 如果您有 cloudera,请检查版本并下载 jars。 示例 CDH 5.9.1 hadoop-common-2.6.0-cdh5.9.1.jar hive-jdbc-1
  • rJava 安装失败(MacOS、Red Hat)(rJava fails to install (MacOS, Red Hat))
    问题 我遇到了许多其他人似乎曾经遇到过的相同的 rJava 问题。 R CMD INSTALL测试JNI程序是否可以编译时出现错误。 输出建议运行R CMD javareconf -ed或sudo R CMD javareconf后者的输出是: computer: home$ sudo R CMD javareconf Java interpreter : /usr/bin/java Java version : 1.6.0_31 Java home path : /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home Java compiler : /usr/bin/javac Java headers gen.: /usr/bin/javah Java archive tool: /usr/bin/jar Java library path: JNI linker flags : -framework JavaVM JNI cpp flags : -I$(JAVA_HOME)/include Updating Java configuration in /Library/Frameworks/R.framework/Resources Done. 之后尝试安装 rJava 时, computer
  • 列出R软件包依赖项而不安装软件包(Listing R Package Dependencies Without Installing Packages)
    问题 是否有一种简单的方法来获取给定软件包的R软件包依赖性(所有递归依赖性)列表,而无需安装软件包及其依赖性? 类似于portupgrade或apt中的假安装。 回答1 您可以使用available.packages函数的结果。 例如,查看ggplot2依赖什么: pack <- available.packages() pack["ggplot2","Depends"] 这使 : [1] "R (>= 2.14), stats, methods" 请注意,根据您要实现的目标,您可能还需要检查Imports字段。 回答2 另一个简洁明了的解决方案是库packrat的内部函数recursivePackageDependencies 。 但是,该软件包必须安装在计算机上的某些库中。 优点是它也可以与自制的非CRAN软件包一起使用。 例子: packrat:::recursivePackageDependencies("ggplot2",lib.loc = .libPaths()[1]) 给予: [1] "R6" "RColorBrewer" "Rcpp" "colorspace" "dichromat" "digest" "gtable" [8] "labeling" "lazyeval" "magrittr" "munsell" "plyr" "reshape2" "rlang"
  • 在 Mac 上安装 rJava 时遇到问题; JNI 数据类型的问题(Trouble installing rJava on Mac ; trouble with JNI data types)
    问题 解决!! 请参阅下面的解决方案,以防此问题将来对其他人有所帮助。 错误 正如标题所描述的那样,它对我来说失败了。 当我尝试从 R 内部安装install.packages('rJava')时,我收到的错误是(有一些前面的良好检查): checking whether JNI programs can be compiled... yes checking JNI data types... configure: error: One or more JNI types differ from the corresponding native type. You may need to use non-standard compiler flags or a different compiler in order to fix this. ERROR: configuration failed for package ‘rJava’ * removing ‘/usr/local/lib/R/3.2/site-library/rJava’ 修复尝试 我已经检查并尝试了以下解决方案,但没有一个对我有用: 须藤 R CMD javareconf 更详细的 sudo R CMD javareconf 导出 JAVA_HOME,但是在这种情况下,我没有执行 LD_LIBRARY_PATH
  • R: JDBC() not finding Java drivers path when connecting to Teradata
    I'm trying to connect to Teradata through RStudio, but for some reason JDBC function has problems recognizing the path where Java drivers sit. See the code below: library(RODBC) library(RJDBC) library(rJava) # both Java drivers definitely exist file.exists('/Users/KULMAK/Documents/TeraJDBC__indep_indep.16.10.00.03/tdgssconfig.jar') [1] TRUE file.exists('/Users/KULMAK/Documents/TeraJDBC__indep_indep.16.10.00.03/terajdbc4.jar') [1] TRUE But when I paste those paths in JDBC call... # allow more elaborated error messages to appear .jclassLoader()$setDebug(1L) drv = JDBC("com.teradata.jdbc
  • rJava loading in R(3.4.1) with OS Sierra
    I have been around a problem with rJava in the newest version of R(3.4.1), Mac (10.12.5) and Java VM (1.8.0_131, x86_64:) Problem: I try it to follow this guide scottdhoover.wordpress.com/2013/03/05/a-basic-rjava-example/ To be able to run rJava in R. Some code from r: library(rJava) .jinit("") .jclassPath() [1] "/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/java" s <- .jnew("java/lang/String", "Hello World!") .jcall(s, "I", "length") [1] 12 So is actually working.. but when I try with my files: .jaddClassPath("/Users/Camilo/Desktop/20170711_Java2R") .jclassPath() [1] "
  • Trouble installing rJava on Mac ; trouble with JNI data types
    RESOLVED!! See my solution below, in case this question helps others in the future. Error As the title described, it it failing for me. The error I'm receiving, when I attempt to install.packages('rJava') from inside R is (with some preceding good checks): checking whether JNI programs can be compiled... yes checking JNI data types... configure: error: One or more JNI types differ from the corresponding native type. You may need to use non-standard compiler flags or a different compiler in order to fix this. ERROR: configuration failed for package ‘rJava’ * removing ‘/usr/local/lib/R/3.2/site
  • 使用 OS Sierra 在 R(3.4.1) 中加载 rJava(rJava loading in R(3.4.1) with OS Sierra)
    问题 我在最新版本的 R(3.4.1)、Mac (10.12.5) 和 Java VM (1.8.0_131, x86_64:) 中遇到了 rJava 的问题 问题: 我尝试按照本指南进行操作 scottdhoover.wordpress.com/2013/03/05/a-basic-rjava-example/ 为了能够在 R 中运行 rJava。 来自 r 的一些代码: library(rJava) .jinit("") .jclassPath() [1] "/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/java" s <- .jnew("java/lang/String", "Hello World!") .jcall(s, "I", "length") [1] 12 所以实际上是有效的..但是当我尝试使用我的文件时: .jaddClassPath("/Users/Camilo/Desktop/20170711_Java2R") .jclassPath() [1] "/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rJava/java" "/Users/Camilo/Desktop /20170711
  • 连接 R 和 Impala(Connect R and Impala)
    问题 我当然知道可重现的示例和代码段,但对于这个问题,我必须(我不能以其他方式)晦涩难懂。 我正在尝试连接 R 和 Impala。 撇开这些问题不谈(“正式地”,我无法在这台 PC 上安装软件......但我已经使用了 R 和 RStudio 的便携式版本) 我试过 RImpala 包。 rimpala.connect(IP = myip, port = the port where Impala sees, principal = maybe this is not clear) 我很确定我的问题的原因是principal论点,我不清楚文档。 无论如何,我已经尝试了文档所说的应该放在那里的几种组合。 在任何情况下,我都会收到相同的错误: .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, 中的错误:java.lang.IllegalArgumentException:Kerberos 主体应该有 3 个部分:10.60.10.22:8888/impala/@tempuser 我在网上搜索过这个错误,它似乎与一些 java 的东西有关,但我对该语言的了解为零。 知道我无法访问我的 PC 可能很有用,例如,我无法安装任何软件或执行只有管理员才能执行的操作。 我知道这个问题写得不好,但正如我这次所说的那样
  • 在Mac / Linux上从R连接到MS SQL Server(Connecting to MS SQL Server from R on Mac/Linux)
    问题 我正在尝试从Mac / Linux上的R连接到Microsoft SQL Server,但是RJDBC出现问题。 当我同时下载了Microsoft的驱动程序和JTDS时,以下行均无效: library(RJDBC) drv <- JDBC("com.microsoft.sqlserver.jdbc.SQLServerDriver", "/Users/victor/Downloads/sqljdbc_3.0/enu/sqljdbc4.jar") drv1 <- JDBC('net.sourceforge.jtds.jdbc.Driver', "/Users/victor/Downloads/jtds-1/jtds-1.3.0.jar") 每个返回以下错误消息: Error in .jfindClass(as.character(driverClass)[1]) : class not found 我怀疑问题可能出在Java版本错误:我的默认Java是 $ java -version java version "1.7.0_11" Java(TM) SE Runtime Environment (build 1.7.0_11-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)
  • 如何使用R从Cassandra读取数据?(How to read data from Cassandra with R?)
    问题 我正在使用R 2.14.1和Cassandra 1.2.11,我有一个单独的程序,该程序已将数据写入单个Cassandra表中。 我无法从R中读取它们。 Cassandra模式的定义如下: create table chosen_samples (id bigint , temperature double, primary key(id)) 我首先尝试过RCassandra软件包(http://www.rforge.net/RCassandra/) > # install.packages("RCassandra") > library(RCassandra) > rc <- RC.connect(host ="192.168.33.10", port = 9160L) > RC.use(rc, "poc1_samples") > cs <- RC.read.table(rc, c.family="chosen_samples") 连接似乎成功,但是将表解析为数据帧失败: > cs Error in data.frame(..dfd. = c("@\"ffffff", "@(<cc><cc><cc><cc><cc><cd>", : duplicate row.names: 我还尝试过使用JDBC连接器,如下所述:http://www.datastax.com/dev
  • libjvm.so:无法打开共享对象文件:没有这样的文件或目录(libjvm.so: cannot open shared object file: No such file or directory)
    问题 尝试使用此代码在我的14.10 ubuntu上的R上安装rJava软件包 if (!require(rJava)) install.packages('rJava') 重新安装jdk之后,遇到了与Java相关的一些错误,包括jni问题,并尝试再次运行相同的代码,但出现了该错误 Error : .onLoad failed in loadNamespace() for 'rJava', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/usr/local/lib/R/site-library/rJava/libs/rJava.so': libjvm.so: cannot open shared object file: No such file or directory Error: loading failed Execution halted ERROR: loading failed * removing ‘/usr/local/lib/R/site-library/rJava’ Warning in install.packages : installation of package ‘rJava’ had non-zero exit
  • rJava load error in RStudio/R after “upgrading” to OSX Yosemite
    I recently "upgraded" from OSX Mountain Lion to Yosemite and from R 3.1.3 to 3.2. Immediately after the upgrade, when I opened R or RStudio I got a pop-up message saying that I needed to install Java 6. In addition, loading rJava or any package that depends on rJava (e.g., xlsx) caused RStudio to crash (R also crashed when I tried this by opening R.app directly). After trying a few fixes found on Stack Overflow and elsewhere (more details below), I am at a point where loading rJava or any package that depends on rJava no longer causes R to crash, but results in the following error: library
  • Connecting to MS SQL Server from R on Mac/Linux
    I'm trying to connect to a Microsoft SQL Server from R on Mac/Linux, and I have problems with RJDBC. When I've downloaded both Microsoft's driver and JTDS, but none of the following lines work: library(RJDBC) drv <- JDBC("com.microsoft.sqlserver.jdbc.SQLServerDriver", "/Users/victor/Downloads/sqljdbc_3.0/enu/sqljdbc4.jar") drv1 <- JDBC('net.sourceforge.jtds.jdbc.Driver', "/Users/victor/Downloads/jtds-1/jtds-1.3.0.jar") Each one returns the following error message: Error in .jfindClass(as.character(driverClass)[1]) : class not found I suspect the issue may be with the wrong Java version: my
  • R错误:java.lang.OutOfMemoryError:Java堆空间(R Error: java.lang.OutOfMemoryError: Java heap space)
    问题 我正在尝试将R连接到Teradata,以将数据直接拉到R中进行分析。 但是,我得到了一个错误, Error in .jcall(rp, "I", "fetch", stride, block) : java.lang.OutOfMemoryError: Java heap space 我尝试通过以下方法设置我的R选项以增加JVM的最大堆大小: options(java.parameters = "-Xmx8g") 我也尝试过使用rJava函数.jinit初始化java参数为: .jinit(parameters="-Xmx8g") 。 但是还是失败了。 计算出的数据大小应约为3G(实际上小于3G)。 回答1 您需要确保在加载rJava或任何其他软件包之前分配了更多的内存。 首先擦除环境(通过rm(list = ls()) ),如有必要,请重新启动R / Rstudio,然后在脚本的开头修改选项。 options(java.parameters = "-Xmx8000m") 参见例如https://support.snowflake.net/s/article/solution-using-r-the-following-error-is-returned-javalangoutofmemoryerror-gc-overhead-limit-exceeded 回答2
  • rjava 依赖包安装 Segmentation fault (core dumped)(rjava dependent package installation Segmentation fault (core dumped))
    问题 我正在尝试重新安装我以前能够安装和使用的软件包。 在我的计算机意外重启后,我正在构建自己的包,然后我开始在加载 rpgraph 包时遇到问题。 所以我决定卸载它并重新安装它。 当我这样做时,我收到以下错误: library(devtools) library(rJava) install_github("Albluca/rpgraph") Downloading GitHub repo Albluca/rpgraph@master from URL https://api.github.com/repos/Albluca/rpgraph/zipball/master Installing rpgraph Running command /usr/lib/R/bin/R Arguments: CMD INSTALL /tmp/Rtmp5OrtLL/devtools505a703b3ccd/Albluca-rpgraph-de04f96 --library=/home/gonzalo/R/x86_64-pc-linux-gnu-library/3.4 --install-tests installing source package ‘rpgraph’ ... ** R ** data *** moving datasets to lazyload DB ** inst **
  • 加载rJava时出错(Error while loading rJava)
    问题 我想加载rJava时出现错误。 已安装JDK。 (我在CentOS VM(cloudera演示vm cdh3u4)上运行R) > library(rJava) Error : .onLoad failed in loadNamespace() for 'rJava', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/home/cloudera/R/x86_64-redhat-linux-gnu-library/2.15/rJava/libs/rJava.so': libjvm.so: cannot open shared object file: No such file or directory Error: package/namespace load failed for ‘rJava’ LD_LIBRARY_PATH设置有问题吗? 如果是,我该如何解决? 我需要运行rJava以便稍后安装rhdfs。 一些更多信息(如果需要): [cloudera@localhost ~]$ java -version java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6
  • R错误:.onLoad在'rJava'的loadNamespace()中失败(R - Error : .onLoad failed in loadNamespace() for 'rJava')
    问题 加载rJava包时,出现以下错误: Error : .onLoad failed in loadNamespace() for 'rJava', details: call: inDL(x, as.logical(local), as.logical(now), ...) error: unable to load shared object 'C:/Users/ankitagarwal5/Documents/R/win-library/3.2/rJava/libs/x64/rJava.dll': LoadLibrary failure: %1 is not a valid Win32 application. In addition: Warning message: package 'rJava' was built under R version 3.2.3 Error: package or namespace load failed for 'rJava' 我引用了StackOverflow上的各种帖子,并尝试进行相应的更改,但是没有任何效果。 这些是我解决该问题所遵循的各种步骤: if (Sys.getenv("JAVA_HOME")!="") Sys.setenv(JAVA_HOME="") library(rJava) 这没用。 然后我尝试了这个:
  • R使用RJDBC写表到Hive(R using RJDBC write table to Hive)
    问题 我已经使用rjdbc成功连接了本地 R3.1.2(win7 64bit rstudio) 和远程配置单元服务器, library(RJDBC) .jinit() dir = "E:/xxx/jars/hive/" for(l in list.files(dir)) { .jaddClassPath(paste( dir ,l,sep="")) } options( java.parameters = "-Xmx8g" ) drv <- JDBC("org.apache.hadoop.hive.jdbc.HiveDriver", "E:/xxx/jars/hive/hive-jdbc-0.11.0.jar") conn <- dbConnect(drv, "jdbc:hive://10.127.130.162:10002/default", "", "" ) dbGetQuery(conn, "select * from test.test limit 10 ") 成功从 hive 读取数据,但我无法使用写入 R 数据帧 dbWriteTable : data(iris) dbWriteTable(conn, iris , "test.dc_test") 错误返回: Error in .jcall(md, "Ljava/sql/ResultSet;", "getTables"
  • Error while loading rJava
    I get an error when I'd like to load rJava. JDK is installed. (I run R on a CentOS VM (cloudera demo vm cdh3u4)) > library(rJava) Error : .onLoad failed in loadNamespace() for 'rJava', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/home/cloudera/R/x86_64-redhat-linux-gnu-library/2.15/rJava/libs/rJava.so': libjvm.so: cannot open shared object file: No such file or directory Error: package/namespace load failed for ‘rJava’ Is there something wrong with LD_LIBRARY_PATH settings? If yes, how can I fix that? I need rJava running that to install rhdfs