Scala profile

Its older brother, Java, has been sweeping the market, making Scala a relatively unknown language. Until the Internet also began to undergo a “century of change” : go, Kotlin, Python and other languages rose to prominence…….

(Photo from Internet)

With the advent of big data, Scala is in the ascendance. The Spark framework has become the de facto standard framework for big data computing, and Spark’s native development language is the unknown Scala. Why did Spark developers choose Scala as their development language?

Both the name and the design concept of Scala come from: Scalable. It integrates the idea of declarative programming and functional programming, and at the same time carries forward the original OOP idea, greatly improving the development efficiency of big data computing.

One more context: Java still dominates big data development: for example, to use the various technologies in the Hadoop ecosystem (Kafka, ZooKeeper, and other projects associated with the Apache Foundation, etc), one of the most basic requirements is to configure the JDK.

But the downside of Java is obvious: it’s too big. (This has changed since Java 8 was released in 2013, and the JDK has now been updated to version 14, with a fair amount of new content.) Python may need only one sentence to complete a function, whereas Java requires a lot of logic written…… Obviously writing a computing framework in Java requires very, very long code.

Spark’s developers were looking for a development tool that would work with other big data frameworks running on the JVM while keeping the code simple and efficient. In this case, the language for this is Scala.

Scala is also standing on the shoulders of giants: it can use the wheels left behind by its older brother Java. It also provides new ideas for Java: Java 8, for example.

Why Spark chose Scala?

Scala’s relationship to Java

  1. In addition to Scala’s own unique syntax, it also supports some Java syntax.

  2. The SDK relies on the JDK to run on top of it.

  3. Scala promotes functional programming: it introduces the concepts of partial functions, currization of functions, and higher-order functions, which are Scala’s first-class citizens.

  4. At the bottom, all Scala code is really a wrapper around Java classes/interfaces.

  5. Both Scala and the Java 8 compiler are the work of Dean Odsky, so you’ll see a bit of Scala in Java 8.

  6. Scala takes OOP to the extreme! Its “basic” data types are also objects and provide many of the same convenience methods as Java’s wrapper classes.

  7. Java has two commands: javac and Java, which are used to compile and run, respectively. Scala also has two corresponding commands: scalac and Scala.

Features of the Scala language

Unlike interpreted languages like Python and javaScript, Scala, like Java, is a statically typed programming language. Compile and then run on the JVM.

Scala is a multi-paradigm programming language, Scala supports both object-oriented and functional programming, and functions are greatly enhanced.

Scala borrows design ideas from Java while incorporating functional programming into Java.

Scala Environment Installation

Scala relies on the JDK to run. And the VERSION number of SDK needs to be corresponding to it, please note.

The Windows platform

  1. SCALA_HOME and PATH are automatically configured according to the installation directory. But be careful!! The installation path must not contain Spaces or Chinese characters. The default installation path of common programs is C:/ Program file, which contains Spaces and is prone to errors.

  2. Alternatively, you can download the ZIP package and decompress it to the corresponding directory. However, you need to manually configure SCALA_HOME and PATH. The configuration method is the same as JDK.

Linux platform

  1. To install Scala in LInux (CENTOS), download the TGZ package from the official website, send it to the VM, decompress the package, and configure it.
#Added the following configuration ----------- to --------profile#Export SCALA_HOME={installation directory} export PATH= $PATH:$SCALA_HOME/bin#------------ Restart the configuration -------------- -------#
source /etc/profile
Copy the code

Scala development tools

In fact, a big data project is often a hybrid of Java and Scala, so Scala’s development tool is still Jet Brains’ IntelliJ IEDA.

However, IDEA does not support Scala development by default, so you need to import the Scala Plugin. Download the corresponding version from the Settings page of IDEA.

The address of the Jet Brains plug-in is as follows. You can download it to your local PC and select Install from the disk.

Jet Brains plugin official website

Scala code development takes place in a Windows environment.

Create the first Scala program starting with “Hello Scala.

Feel the difference between Scala’s main function and Java’s main function:

object HelloScala {
 // Define main function (parameter: array [String]):unit = {struct}
 def main(args:Array[String]): unit ={

 }
}
//Java type:
//public class HelloJava {
// public static void main(String[] args) {
// System.out.println("Hello Java!" );
/ /}
/ /}
Copy the code

Object in Scala means that it is an associated object (defined in a later chapter), and HelloScala is the name of the associated object.

The underlying implementation of the accompanying object is HelloScala$. To observe this, we use NotePad to edit a Scala file on Windows and compile it with the original Scalac, resulting in two.class files:

To see the contents of the two files and the differences, we decompilate the class file using jD-gui.exe:

//----------------------HelloScala.class------------------------//
import scala.reflect.ScalaSignature;

// Ignore some code
public final class HelloScala
{
  public static void main(String[] paramArrayOfString)
  {
    HelloScala..MODULE$.main(paramArrayOfString);
  }
}
//----------------------HelloScala$.class-----------------------//
import scala.Predef$;

public final class HelloScala$
{
  public static final  MODULE$;

  static
  {
    new(a); }public void main(String[] args)
  {
    Predef$.MODULE$.println("Hello Scala!");
  }

  private HelloScala$()
  {
    MODULE$ = this; }}Copy the code

Looking at the compiled code, we can see that what is essentially the main function in HelloScala.class is actually the main instance method that wraps the MODULE$in HelloScala$. Class.

How to use JD-GUI to view the.class file code, I have mentioned before.

Why is it that when you open a.class file with TXT, it’s a mess, but when you open a.java file, it’s a mess?

Why go through such a packaging process?

Martin Odsky, the designer of Scala, distinguishes between the static and non-static content of a program, arguing that static content should not be the domain of OOP objects.

For example, the static variable Pi of the Circle class has nothing to do with the actual Circle object.

So the designer declares the non-static part of the same class as class: associated class, and its static part as Object: associated object. (This part will be added in the follow-up study)

HelloScala’s main function declaration

The def keyword declares a method. If the method is declared to be the main function (the entry to the program), the method (function) name must be declared main. This is a rule, such as Java’s main function must be (Public static void main()).

Scala’s declared variables feature parameter names first and type second.

Array[String] indicates that this is a String Array. Scala’s [] brackets do what Java’s <> does: they refer to generics.

def main(args:….) :Unit={… } :Unit = denotes that the return value of the function structure is void. Unit is a real class.

Scala and Java compilers have a lot in common and a lot in common. For example, you can run a Javac-compiled.class file using a Scala command, but running a Scalac-compiled.class file using a Java command may not be successful.

Note that the class running the main function needs to be declared object, not class. Just as Java’s main functions run in a static domain, Scala’s main functions run in a static domain, and in Scala, associated objects hold the static parts of a class.

Use Java classes to mimic Scala’s mechanics

Review two class modifiers in Java

Before we do that, let’s review two class modifiers for Java classes: public, or default. A. Java file can have multiple classes, or even none (for example, an empty file is created, but nothing is written).

At most one. Java file can be declared as a public class. The name declared as a public class must be the same as the name of the.java file, and there is no restriction on other non-public classes in the same.Java file.

public class PublicClazz {
    public static void main(String[] args) {}}class defaultCLazz{}Copy the code

Multiple classes declared in the same.java file can be treated as if they are under the same package and can call each other. The public property can also be called with other class objects in the.java file. But they are not visible to external packages.

At compile time, Javac splits multiple classes in the same. Java into multiple. Class files. All classes except those marked public in a. Java file are compiled to default.

By convention, however, there is only one class of the same name encapsulated in a.Java file, which makes the overall architecture more explicit.

Use Java code to duplicate Scala’s underlying code

To restore Scala’s wrapping process, use Java code, which is a helloScala.java file:

package simulation;

// HelloScala class for external use.
public class HelloScala {
 public static void main(String[] args) { HelloScala$.MODULE$.main(args); }}class HelloScala${
 static final HelloScala$ MODULE$;
 // Initialize it in the static field.
 static {
   MODULE$ = new HelloScala$();
 }
   
 public void main(String[] args){
   System.out.println("The main function that is actually executed."); }}Copy the code

This Java program restores 90% of the scala.class file.

What this program really does is execute the main method of MODULE$. Note that MODULE$is static, final. Instances of HelloScala companion objects, which essentially maintain the same static instance of MODULE$, conform to the singleton design.

In Scala, Java’s static functionality is implemented by placing variables in a singleton. This singleton class is called an associated object in Scala and is decorated with a separate object keyword instead of class.

The following figure shows that HelloScala, a companion object of Scala, consists of helloScala.class and HelloScala$. Class.

Refer to the design patterns in the Rookie tutorial:

Rookie tutorial: Design mode

How Scala works

  1. . Scala files are compiled into. Class files by Scalac and then run.

  2. . Scala files are run directly by Scala, but are slow and do not generate. Class files. (We hardly ever do.)

Scala considerations

  1. Strictly case sensitive.

  2. A semicolon is not required if a statement is in a line. If there are multiple statements in a line, a semicolon (;) must be added between the middle statements.

  3. The source files have an extension name of.scala, but underneath they are compiled into.class files.

  4. Scala’s program execution entry is the main function, which runs in the accompanying object.

Scala string output

Scala supports three ways to output strings. In addition to Java and C-style output, linux-style references are also referenced. Note the details: if a ‘\’ is used as a reference, the string must be preceded by an S identifier.

var string1 : String = "Hello"
var string : String = "Scala"

// The same as system.out.println in Java.
println(string1 + "," + string)

// C formatted output
printf("%s,%s",string1,string)

//Linux $reference
printf(s"$string.$string1")
Copy the code

How do I view Scala’s associated source code?

When the Scala development plug-in and SDK were just installed, the source code was not associated, so it could not be seen during programming. The underlying implementation of Scala is quite troublesome. Here, associate the source code.

Hold Down Ctrl and click on any Scala class to trace the declaration file for that class, and IDEA prompts Sources not found.

Download the scala-sources-2.xx.x.tar folder from the official website and decompress it to the lib directory of the SDK. Note that the version must be the same. Then select Associate from Attach Sources.

You can confirm the version with the Scala-version command before installation.

Scala document comments

Start with doc style comments in a. Scala program:

object ScalaDocTest {

  /** * This is the main function declaration in Scala. The main function must be declared in the companion object object. * @param args runtime argument */
  def main(args: Array[String) :Unit = {
    println("This is a main function.")}/** * This method is static because it is declared in the companion object ScalaDocTest. * @return returns a" Hello Scala" string. * /
  def greet() :String= {return "Hello Scala"}}Copy the code

You can open Terminal under IDEA, which is the root directory of the project by default. Go to the scala file directory and run the following command to select all scala files in a directory:

scaladoc -d{destination path} {scala file path}Copy the code

The command line prompts: Model contains X ducumentable templates. Represents a place at X where the document can be generated and generates.

Document annotations in Scala files are extracted to HTML format.