Lines with a: 60, Lines with b: 29 Posted on JSeptemAuthor nanxiao Categories Technology Tags Apache-Spark, Intellij, Scala using builtin-java classes where applicable C :\ Work\ Intellij_scala\ CountWord\out\artifacts\ CountWord_jar\ CountWord.jarġ5/ 06/ 17 17 : 05 : 51 WARN NativeCodeLoader : Unable to load native-hadoop library fo 4\bin>spark-submit - class " CountWord" - master local Then using spark-submit command execute jar package: C :\spark- 1.3. “ File” -> “ Project Structure” -> “ Artifacts“, then select options like this: (4) Add spark package and create an scala object in “ src -> main -> scala-2.10” folder, the final file layout likes this: (5) Run it! (3) In Windows, modify the scala version to 2.10.4 in build.sbt: (2) Fill the “ project name” and “ project location“, then click “ Finish“: (1) Select “ New project” -> “ Scala” -> “ SBT“, then click “ Next“: The following part introduces creating SBT project:
F or detail information, you should refer the following posts:Ī) Apache Spark checkpoint issue on windows
Intellij jar executable download#
Please notice “ tProperty("", "c:\\winutil\\")”, You should download winutils.exe and put it in the folder: C:\winutil\bin. Println( "Lines with a: %s, Lines with b: %s".format(numAs, numBs)) Val numBs = logData.filter(line => ntains( "b")).count() Val numAs = logData.filter(line => ntains( "a")).count() Val logData = sc.textFile(logFile, 2).cache() Val conf = new SparkConf().setAppName( "Simple Application").setMaster( "local") Val logFile = "C:\\spark-1.3.1-bin-hadoop2.4\\README.md" // Should be some file on your system (6) Write a simple CountWord application: import (3) Select spark-assembly-x.x.x-hadoopx.x.x.jar, press OK: (2) Select “ Modules” -> “ Dependencies” -> “ +” -> “ Library” -> “ Java“: (1) Select “ File” -> “ Project Structure“:
Intellij jar executable install#
Secondly, you should install Scala plugin and create a Scala project, you can refer this document: Getting Started with Scala in IntelliJ IDEA 14.1.Īfter all the above steps are done, the project view should like this: ActorCell $.( ActorCell.scala)Īt akka.actor.
HashSet $.empty() Lscala/collection/immutable/ HashSet Īt akka.actor. My Operating System is Windows 7, so this tutorial may be little difference for your environment.įirstly, you should install Scala 2.10.x version on Windows to run Spark, else you would get errors like this: Exception in thread "main" java.lang. Yongjun Cho on gcc’s enable “–enable-default-pie” option make you stuck at “relocation R_X86_64_32S against …” error.Danil on Fix “Permission denied rb_file_s_symlink” error of installing homebrew.Jake on Configure static IP address on Arch Linux.
Intellij jar executable how to#