Package us.kbase.common.service
Class JsonTokenStream
- java.lang.Object
-
- com.fasterxml.jackson.core.JsonParser
-
- us.kbase.common.service.JsonTokenStream
-
- All Implemented Interfaces:
Versioned,java.io.Closeable,java.lang.AutoCloseable
public class JsonTokenStream extends JsonParser
This class gives a way to keep json data in different types of source (file, string or byte array) and to process this data through sequence of json tokens. The difference of this class comparing to standard UTF8StreamJsonParser or ReaderBasedJsonParser is that here we can deal with large text values since they are processed as whole tokens in standard jackson approach. Here we substitute these large text values by special keywords storing mapping from these keywords to position of real text in character stream. This mapping is stored in largeStringPos. For that we wrap input reader before parsing it into tokens by wrapper searching these large texts (see getWrapperForLargeStrings method for Reader for details). And during writing operation we wrap output writer by wrapper substituting these keywords back into large strings (see getWrapperForLargeStrings for Writer). For searching large string by stored position LargeStringSearchingReader is used by calling getLargeStringReader method. Another useful feature which is not present in standard parsers is tracking current json path. Even more, you can define json path as a root point for your token stream and after that this stream will provide tokens only from subtree of your json data deeper than root path. This feature is important for UObject creation based on fragment in bigger json data covered by JsonTokenStream.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interfaceJsonTokenStream.DebugOpenCloseListenerUseful for listening data stream open/close events for debug purposes.-
Nested classes/interfaces inherited from class com.fasterxml.jackson.core.JsonParser
JsonParser.Feature, JsonParser.NumberType
-
-
Field Summary
-
Fields inherited from class com.fasterxml.jackson.core.JsonParser
_features, _requestPayload
-
-
Constructor Summary
Constructors Constructor Description JsonTokenStream(java.lang.Object data)Create token stream for data source of one of the following types: File, String, byte[], JsonNode.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description booleancanUseSchema(FormatSchema arg0)voidclearCurrentToken()voidclose()JsonParserconfigure(JsonParser.Feature arg0, boolean arg1)java.io.ReadercreateDataReader()Create reader for data source.JsonParserdisable(JsonParser.Feature arg0)JsonParserenable(JsonParser.Feature arg0)java.math.BigIntegergetBigIntegerValue()byte[]getBinaryValue()byte[]getBinaryValue(Base64Variant arg0)booleangetBooleanValue()bytegetByteValue()ObjectCodecgetCodec()intgetCopyBufferSize()Gets the size of the buffer used when copying the data contained in this JTS directly to an output stream or writer.JsonLocationgetCurrentLocation()java.lang.StringgetCurrentName()java.util.List<java.lang.String>getCurrentPath()Json path before current token.JsonTokengetCurrentToken()intgetCurrentTokenId()java.math.BigDecimalgetDecimalValue()doublegetDoubleValue()java.lang.ObjectgetEmbeddedObject()java.nio.charset.CharsetgetEncoding()Get the encoding of the data source, if any.floatgetFloatValue()protected JsonParsergetInner()Standard json parser used inside this parser.java.lang.ObjectgetInputSource()intgetIntValue()java.io.ReadergetLargeStringReader(java.lang.String text)Search large string position and allow to read this large string as reader.JsonTokengetLastClearedToken()longgetLongValue()JsonParser.NumberTypegetNumberType()java.lang.NumbergetNumberValue()JsonStreamContextgetParsingContext()FormatSchemagetSchema()shortgetShortValue()java.lang.StringgetText()char[]getTextCharacters()intgetTextLength()intgetTextOffset()JsonLocationgetTokenLocation()booleangetValueAsBoolean()booleangetValueAsBoolean(boolean arg0)doublegetValueAsDouble()doublegetValueAsDouble(double arg0)intgetValueAsInt()intgetValueAsInt(int arg0)longgetValueAsLong()longgetValueAsLong(long arg0)java.lang.StringgetValueAsString()java.lang.StringgetValueAsString(java.lang.String arg0)booleanhasCurrentToken()booleanhasTextCharacters()booleanhasToken(JsonToken t)booleanhasTokenId(int id)booleanhasTrustedWholeJson()Returns true if this JTS has been marked as wrapping data that is known good JSON.booleanisClosed()booleanisEnabled(JsonParser.Feature arg0)booleanisExpectedStartArrayToken()booleanisLargeString(java.lang.String text)Check if this text is keyword mapping to position of real large string.java.lang.BooleannextBooleanValue()java.lang.StringnextFieldName()booleannextFieldName(SerializableString str)intnextIntValue(int arg0)longnextLongValue(long arg0)java.lang.StringnextTextValue()JsonTokennextToken()JsonTokennextValue()voidoverrideCurrentName(java.lang.String arg0)intreadBinaryValue(Base64Variant arg0, java.io.OutputStream arg1)intreadBinaryValue(java.io.OutputStream arg0)<T> TreadValueAs(TypeReference<?> arg0)<T> TreadValueAs(java.lang.Class<T> arg0)<T extends TreeNode>
TreadValueAsTree()<T> java.util.Iterator<T>readValuesAs(TypeReference<?> arg0)<T> java.util.Iterator<T>readValuesAs(java.lang.Class<T> arg0)intreleaseBuffered(java.io.OutputStream arg0)intreleaseBuffered(java.io.Writer arg0)booleanrequiresCustomCodec()voidsetCodec(ObjectCodec arg0)JsonTokenStreamsetCopyBufferSize(int size)Sets the size of the buffer used when copying the data contained in this JTS directly to an output stream or writer.static voidsetDebugOpenCloseListener(JsonTokenStream.DebugOpenCloseListener debugOpenCloseListener)Useful for listening data stream open/close events for debug purposes.JsonTokenStreamsetRoot(java.util.List<java.lang.String> root)Define root point in data source from which token stream should start.voidsetSchema(FormatSchema arg0)JsonTokenStreamsetTrustedWholeJson(boolean twj)Specify that this JTS wraps data that is known good JSON.JsonParserskipChildren()Versionversion()voidwriteJson(java.io.File f)Write all selected (probably through setRoot) tokens into output file.voidwriteJson(java.io.OutputStream os)Write all selected (probably through setRoot) tokens into output stream.voidwriteJson(java.io.Writer w)Write all selected (probably through setRoot) tokens into output writer.voidwriteTokens(JsonGenerator jgen)Write all selected (probably through setRoot) tokens into output generator.-
Methods inherited from class com.fasterxml.jackson.core.JsonParser
_codec, _constructError, _reportUnsupportedOperation, canParseAsync, canReadObjectId, canReadTypeId, currentName, currentToken, currentTokenId, finishToken, getCurrentValue, getFeatureMask, getFormatFeatures, getNonBlockingInputFeeder, getObjectId, getText, getTypeId, isExpectedStartObjectToken, isNaN, overrideFormatFeatures, overrideStdFeatures, setCurrentValue, setFeatureMask, setRequestPayloadOnError, setRequestPayloadOnError, setRequestPayloadOnError
-
-
-
-
Constructor Detail
-
JsonTokenStream
public JsonTokenStream(java.lang.Object data) throws JsonParseException, java.io.IOExceptionCreate token stream for data source of one of the following types: File, String, byte[], JsonNode.- Parameters:
data-- Throws:
JsonParseExceptionjava.io.IOException
-
-
Method Detail
-
getEncoding
public java.nio.charset.Charset getEncoding()
Get the encoding of the data source, if any.- Returns:
- The data source endoding. Returns null if the data source is a String.
-
setTrustedWholeJson
public JsonTokenStream setTrustedWholeJson(boolean twj)
Specify that this JTS wraps data that is known good JSON. Cannot be set as true if the root is not at /, and will be set to false if the root is set to a location other than /. The effect of this parameter is that the wrapped data is written directly to the output stream or writer, bypassing parsing the JSON (and thus checking correctness), for a substantial speed increase.- Parameters:
twj- whether this object contains known good JSON.- Returns:
- this JTS
-
hasTrustedWholeJson
public boolean hasTrustedWholeJson()
Returns true if this JTS has been marked as wrapping data that is known good JSON. See setTrustedWholeJson for more details.- Returns:
- true if this JTS has been marked as wrapping data that is known good JSON.
-
setCopyBufferSize
public JsonTokenStream setCopyBufferSize(int size)
Sets the size of the buffer used when copying the data contained in this JTS directly to an output stream or writer.- Parameters:
size- the size of the buffer.- Returns:
- this JTS
-
getCopyBufferSize
public int getCopyBufferSize()
Gets the size of the buffer used when copying the data contained in this JTS directly to an output stream or writer.- Returns:
- the size of the buffer.
-
setRoot
public JsonTokenStream setRoot(java.util.List<java.lang.String> root) throws JsonParseException, java.io.IOException
Define root point in data source from which token stream should start.- Parameters:
root- list of fields or array numbers defining json path- Returns:
- this JTS
- Throws:
JsonParseExceptionjava.io.IOException
-
getCurrentPath
public java.util.List<java.lang.String> getCurrentPath() throws java.io.IOExceptionJson path before current token.- Returns:
- list of fields or array numbers defining json path
- Throws:
java.io.IOException
-
getInner
protected JsonParser getInner()
Standard json parser used inside this parser.- Returns:
- original parser wrapped by this JTS
-
createDataReader
public java.io.Reader createDataReader() throws java.io.IOExceptionCreate reader for data source.- Returns:
- Reader
- Throws:
java.io.IOException
-
clearCurrentToken
public void clearCurrentToken()
- Specified by:
clearCurrentTokenin classJsonParser
-
close
public void close() throws java.io.IOException- Specified by:
closein interfacejava.lang.AutoCloseable- Specified by:
closein interfacejava.io.Closeable- Specified by:
closein classJsonParser- Throws:
java.io.IOException
-
getBigIntegerValue
public java.math.BigInteger getBigIntegerValue() throws java.io.IOException, JsonParseException- Specified by:
getBigIntegerValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getBinaryValue
public byte[] getBinaryValue(Base64Variant arg0) throws java.io.IOException, JsonParseException
- Specified by:
getBinaryValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getCodec
public ObjectCodec getCodec()
- Specified by:
getCodecin classJsonParser
-
getCurrentLocation
public JsonLocation getCurrentLocation()
- Specified by:
getCurrentLocationin classJsonParser
-
getCurrentName
public java.lang.String getCurrentName() throws java.io.IOException, JsonParseException- Specified by:
getCurrentNamein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getCurrentToken
public JsonToken getCurrentToken()
- Specified by:
getCurrentTokenin classJsonParser
-
getDecimalValue
public java.math.BigDecimal getDecimalValue() throws java.io.IOException, JsonParseException- Specified by:
getDecimalValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getDoubleValue
public double getDoubleValue() throws java.io.IOException, JsonParseException- Specified by:
getDoubleValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getEmbeddedObject
public java.lang.Object getEmbeddedObject() throws java.io.IOException, JsonParseException- Overrides:
getEmbeddedObjectin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getFloatValue
public float getFloatValue() throws java.io.IOException, JsonParseException- Specified by:
getFloatValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getIntValue
public int getIntValue() throws java.io.IOException, JsonParseException- Specified by:
getIntValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getLastClearedToken
public JsonToken getLastClearedToken()
- Specified by:
getLastClearedTokenin classJsonParser
-
getLongValue
public long getLongValue() throws java.io.IOException, JsonParseException- Specified by:
getLongValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getNumberType
public JsonParser.NumberType getNumberType() throws java.io.IOException, JsonParseException
- Specified by:
getNumberTypein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getNumberValue
public java.lang.Number getNumberValue() throws java.io.IOException, JsonParseException- Specified by:
getNumberValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getParsingContext
public JsonStreamContext getParsingContext()
- Specified by:
getParsingContextin classJsonParser
-
getText
public java.lang.String getText() throws java.io.IOException, JsonParseException- Specified by:
getTextin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getTextCharacters
public char[] getTextCharacters() throws java.io.IOException, JsonParseException- Specified by:
getTextCharactersin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getTextLength
public int getTextLength() throws java.io.IOException, JsonParseException- Specified by:
getTextLengthin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getTextOffset
public int getTextOffset() throws java.io.IOException, JsonParseException- Specified by:
getTextOffsetin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getTokenLocation
public JsonLocation getTokenLocation()
- Specified by:
getTokenLocationin classJsonParser
-
getValueAsString
public java.lang.String getValueAsString(java.lang.String arg0) throws java.io.IOException, JsonParseException- Specified by:
getValueAsStringin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
hasCurrentToken
public boolean hasCurrentToken()
- Specified by:
hasCurrentTokenin classJsonParser
-
hasTextCharacters
public boolean hasTextCharacters()
- Specified by:
hasTextCharactersin classJsonParser
-
isClosed
public boolean isClosed()
- Specified by:
isClosedin classJsonParser
-
nextToken
public JsonToken nextToken() throws java.io.IOException, JsonParseException
- Specified by:
nextTokenin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextValue
public JsonToken nextValue() throws java.io.IOException, JsonParseException
- Specified by:
nextValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
overrideCurrentName
public void overrideCurrentName(java.lang.String arg0)
- Specified by:
overrideCurrentNamein classJsonParser
-
setCodec
public void setCodec(ObjectCodec arg0)
- Specified by:
setCodecin classJsonParser
-
skipChildren
public JsonParser skipChildren() throws java.io.IOException, JsonParseException
- Specified by:
skipChildrenin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
version
public Version version()
- Specified by:
versionin interfaceVersioned- Specified by:
versionin classJsonParser
-
canUseSchema
public boolean canUseSchema(FormatSchema arg0)
- Overrides:
canUseSchemain classJsonParser
-
configure
public JsonParser configure(JsonParser.Feature arg0, boolean arg1)
- Overrides:
configurein classJsonParser
-
disable
public JsonParser disable(JsonParser.Feature arg0)
- Overrides:
disablein classJsonParser
-
enable
public JsonParser enable(JsonParser.Feature arg0)
- Overrides:
enablein classJsonParser
-
getBinaryValue
public byte[] getBinaryValue() throws java.io.IOException, JsonParseException- Overrides:
getBinaryValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getBooleanValue
public boolean getBooleanValue() throws java.io.IOException, JsonParseException- Overrides:
getBooleanValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getByteValue
public byte getByteValue() throws java.io.IOException, JsonParseException- Overrides:
getByteValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getInputSource
public java.lang.Object getInputSource()
- Overrides:
getInputSourcein classJsonParser
-
getSchema
public FormatSchema getSchema()
- Overrides:
getSchemain classJsonParser
-
getShortValue
public short getShortValue() throws java.io.IOException, JsonParseException- Overrides:
getShortValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsBoolean
public boolean getValueAsBoolean() throws java.io.IOException, JsonParseException- Overrides:
getValueAsBooleanin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsBoolean
public boolean getValueAsBoolean(boolean arg0) throws java.io.IOException, JsonParseException- Overrides:
getValueAsBooleanin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsDouble
public double getValueAsDouble() throws java.io.IOException, JsonParseException- Overrides:
getValueAsDoublein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsDouble
public double getValueAsDouble(double arg0) throws java.io.IOException, JsonParseException- Overrides:
getValueAsDoublein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsInt
public int getValueAsInt() throws java.io.IOException, JsonParseException- Overrides:
getValueAsIntin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsInt
public int getValueAsInt(int arg0) throws java.io.IOException, JsonParseException- Overrides:
getValueAsIntin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsLong
public long getValueAsLong() throws java.io.IOException, JsonParseException- Overrides:
getValueAsLongin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsLong
public long getValueAsLong(long arg0) throws java.io.IOException, JsonParseException- Overrides:
getValueAsLongin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
getValueAsString
public java.lang.String getValueAsString() throws java.io.IOException, JsonParseException- Overrides:
getValueAsStringin classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
isEnabled
public boolean isEnabled(JsonParser.Feature arg0)
- Overrides:
isEnabledin classJsonParser
-
isExpectedStartArrayToken
public boolean isExpectedStartArrayToken()
- Overrides:
isExpectedStartArrayTokenin classJsonParser
-
nextBooleanValue
public java.lang.Boolean nextBooleanValue() throws java.io.IOException, JsonParseException- Overrides:
nextBooleanValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextFieldName
public java.lang.String nextFieldName() throws java.io.IOException, JsonParseException- Overrides:
nextFieldNamein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextFieldName
public boolean nextFieldName(SerializableString str) throws java.io.IOException, JsonParseException
- Overrides:
nextFieldNamein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextIntValue
public int nextIntValue(int arg0) throws java.io.IOException, JsonParseException- Overrides:
nextIntValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextLongValue
public long nextLongValue(long arg0) throws java.io.IOException, JsonParseException- Overrides:
nextLongValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
nextTextValue
public java.lang.String nextTextValue() throws java.io.IOException, JsonParseException- Overrides:
nextTextValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
readBinaryValue
public int readBinaryValue(Base64Variant arg0, java.io.OutputStream arg1) throws java.io.IOException, JsonParseException
- Overrides:
readBinaryValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
readBinaryValue
public int readBinaryValue(java.io.OutputStream arg0) throws java.io.IOException, JsonParseException- Overrides:
readBinaryValuein classJsonParser- Throws:
java.io.IOExceptionJsonParseException
-
readValueAs
public <T> T readValueAs(java.lang.Class<T> arg0) throws java.io.IOException, JsonProcessingException- Overrides:
readValueAsin classJsonParser- Throws:
java.io.IOExceptionJsonProcessingException
-
readValueAs
public <T> T readValueAs(TypeReference<?> arg0) throws java.io.IOException, JsonProcessingException
- Overrides:
readValueAsin classJsonParser- Throws:
java.io.IOExceptionJsonProcessingException
-
readValueAsTree
public <T extends TreeNode> T readValueAsTree() throws java.io.IOException, JsonProcessingException
- Overrides:
readValueAsTreein classJsonParser- Throws:
java.io.IOExceptionJsonProcessingException
-
readValuesAs
public <T> java.util.Iterator<T> readValuesAs(java.lang.Class<T> arg0) throws java.io.IOException, JsonProcessingException- Overrides:
readValuesAsin classJsonParser- Throws:
java.io.IOExceptionJsonProcessingException
-
readValuesAs
public <T> java.util.Iterator<T> readValuesAs(TypeReference<?> arg0) throws java.io.IOException, JsonProcessingException
- Overrides:
readValuesAsin classJsonParser- Throws:
java.io.IOExceptionJsonProcessingException
-
releaseBuffered
public int releaseBuffered(java.io.OutputStream arg0) throws java.io.IOException- Overrides:
releaseBufferedin classJsonParser- Throws:
java.io.IOException
-
releaseBuffered
public int releaseBuffered(java.io.Writer arg0) throws java.io.IOException- Overrides:
releaseBufferedin classJsonParser- Throws:
java.io.IOException
-
requiresCustomCodec
public boolean requiresCustomCodec()
- Overrides:
requiresCustomCodecin classJsonParser
-
setSchema
public void setSchema(FormatSchema arg0)
- Overrides:
setSchemain classJsonParser
-
writeTokens
public void writeTokens(JsonGenerator jgen) throws java.io.IOException
Write all selected (probably through setRoot) tokens into output generator.- Parameters:
jgen-- Throws:
java.io.IOException
-
writeJson
public void writeJson(java.io.File f) throws java.io.IOExceptionWrite all selected (probably through setRoot) tokens into output file.- Parameters:
f-- Throws:
java.io.IOException
-
writeJson
public void writeJson(java.io.OutputStream os) throws java.io.IOExceptionWrite all selected (probably through setRoot) tokens into output stream.- Parameters:
os-- Throws:
java.io.IOException
-
writeJson
public void writeJson(java.io.Writer w) throws java.io.IOExceptionWrite all selected (probably through setRoot) tokens into output writer.- Parameters:
w-- Throws:
java.io.IOException
-
isLargeString
public boolean isLargeString(java.lang.String text)
Check if this text is keyword mapping to position of real large string.- Parameters:
text-- Returns:
- was this string substituted as large string or not
-
getLargeStringReader
public java.io.Reader getLargeStringReader(java.lang.String text) throws java.io.IOExceptionSearch large string position and allow to read this large string as reader.- Parameters:
text-- Returns:
- reader substituting large strings
- Throws:
java.io.IOException
-
setDebugOpenCloseListener
public static void setDebugOpenCloseListener(JsonTokenStream.DebugOpenCloseListener debugOpenCloseListener)
Useful for listening data stream open/close events for debug purposes.- Parameters:
debugOpenCloseListener-
-
getCurrentTokenId
public int getCurrentTokenId()
- Specified by:
getCurrentTokenIdin classJsonParser
-
hasToken
public boolean hasToken(JsonToken t)
- Specified by:
hasTokenin classJsonParser
-
hasTokenId
public boolean hasTokenId(int id)
- Specified by:
hasTokenIdin classJsonParser
-
-