IDF
Computes the Inverse Document Frequency (IDF) of a collection of documents.
This operation is ported from Spark ML.
For a comprehensive introduction, see
Spark documentation.
For scala docs details, see
org.apache.spark.ml.feature.IDF documentation.
Since: Seahorse 1.0.0
| Port |
Type Qualifier |
Description |
0 | DataFrame | The input DataFrame. |
Output
| Port |
Type Qualifier |
Description |
0 | DataFrame | The output DataFrame. |
1 | Transformer | A Transformer that allows to apply the operation on other DataFrames using a Transform. |
Parameters
| Name |
Type |
Description |
input column |
SingleColumnSelector |
The input column name. |
output |
SingleChoice |
Output generation mode. Possible values: ["replace input column", "append new column"] |
min documents frequency |
Numeric |
The minimum number of documents in which a term should appear. |
Example
Parameters
| Name |
Value |
input column |
"features" |
output |
append new column |
output column |
"values" |
min documents frequency |
0.0 |
| features |
| [0.0,1.0,0.0,2.0] |
| [0.0,1.0,2.0,3.0] |
| [0.0,1.0,0.0,0.0] |
Output
| features |
values |
| [0.0,1.0,0.0,2.0] |
[0.0,0.0,0.0,0.5753641449035617] |
| [0.0,1.0,2.0,3.0] |
[0.0,0.0,1.3862943611198906,0.8630462173553426] |
| [0.0,1.0,0.0,0.0] |
[0.0,0.0,0.0,0.0] |