Generic Slo: Casting and Chaining Warnings

Closed Issue created 3 years ago by Lorenz Boguhn

The generic slo code throws 2 warinigs/errors in my use case. In short they are chained indexing of values, and casting to integer (by float values). Both arise from the following function:

def aggr_query(values: dict, warmup: int, aggr_func):
    df = pd.DataFrame.from_dict(values)
    df.columns = ['timestamp', 'value']
    filtered = df[df['timestamp'] >= (df['timestamp'][0] + warmup)]
    filtered['value'] = filtered['value'].astype(int)
    return filtered['value'].aggregate(aggr_func)

I suggest to change:

filtered['value'] = filtered['value'].astype(int) to:

filtered.loc[:, 'value'].astype(float).astype(int) (Not tested)

Generic slo log:

Dataframe error:

INFO:     127.0.0.1:53888 - "POST / HTTP/1.1" 200 OK
2022-05-18 13:37:10,458 INFO API: Received request with metadata: {'warmup': 60, 'queryAggregation': 'max', 'repetitionAggregation': 'median', 'operator': 'lte', 'threshold': 100000000}
INFO:     127.0.0.1:53888 - "POST / HTTP/1.1" 500 Internal Server Error
/code/app/./main.py:40: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  filtered['value'] = filtered['value'].astype(int)

pandas doc

Integer casting error:

2022-05-18 13:37:10,657 INFO API: Received request with metadata: {'warmup': 60, 'queryAggregation': 'max', 'repetitionAggregation': 'median', 'operator': 'lte', 'threshold': 100000000}
INFO:     127.0.0.1:53928 - "POST / HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 226, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 159, in run_endpoint_function
    return await dependant.call(**values)
  File "/code/app/./main.py", line 68, in check_slo
    query_results = [aggr_query(r[0]["values"], warmup, query_aggregation) for r in data["results"]]
  File "/code/app/./main.py", line 68, in <listcomp>
    query_results = [aggr_query(r[0]["values"], warmup, query_aggregation) for r in data["results"]]
  File "/code/app/./main.py", line 40, in aggr_query
    filtered['value'] = filtered['value'].astype(int)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line 5698, in astype
    new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 582, in astype
    return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 442, in apply
    applied = getattr(b, f)(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 625, in astype
    values = astype_nansafe(vals1d, dtype, copy=True)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/dtypes/cast.py", line 874, in astype_nansafe
2022-05-18 13:37:10,661 ERROR uvicorn.error: Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 226, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 159, in run_endpoint_function
    return await dependant.call(**values)
  File "/code/app/./main.py", line 68, in check_slo
    query_results = [aggr_query(r[0]["values"], warmup, query_aggregation) for r in data["results"]]
  File "/code/app/./main.py", line 68, in <listcomp>
    query_results = [aggr_query(r[0]["values"], warmup, query_aggregation) for r in data["results"]]
  File "/code/app/./main.py", line 40, in aggr_query
    filtered['value'] = filtered['value'].astype(int)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line 5698, in astype
    new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 582, in astype
    return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 442, in apply
    applied = getattr(b, f)(**kwargs)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 625, in astype
    values = astype_nansafe(vals1d, dtype, copy=True)
  File "/usr/local/lib/python3.8/site-packages/pandas/core/dtypes/cast.py", line 874, in astype_nansafe
    return lib.astype_intsafe(arr.ravel(), dtype).reshape(arr.shape)
  File "pandas/_libs/lib.pyx", line 562, in pandas._libs.lib.astype_intsafe
ValueError: invalid literal for int() with base 10: '27482.000000000007'
    return lib.astype_intsafe(arr.ravel(), dtype).reshape(arr.shape)
  File "pandas/_libs/lib.pyx", line 562, in pandas._libs.lib.astype_intsafe
ValueError: invalid literal for int() with base 10: '27482.000000000007'
2022-05-18 13:57:10,653 INFO API: Received request with metadata: {'warmup': 60, 'queryAggregation': 'max', 'repetitionAggregation': 'median', 'operator': 'lte', 'threshold': 100000000}

Stackoverflow python casting float to int

Activity

Please register or sign in to reply