Posted
over 5 years
ago
by
Dev
I am using SocketIO to stream Twitter data (after due processing in a Spark layer) in order to stream the data to a browser. However, I notice that a socket disconnect event is getting fired randomly saying,
Client is gone, closing
... [More]
socket
Also, I'm using eventlet, and library is monkey patched. On service request, server offloads the entire workflow to a thread as shown in the following snippet, so server is free to process PING/PONG packets.
wf = work_flow.get(current_user.get_id())
if not wf:
wf = work_flow.WorkFlow(current_user.get_id())
work_flow.add(wf)
keys = request.form['search_terms']
interval = request.form['report_interval']
print('Search Terms: ', keys)
print('Report Interval: ', interval)
keys = keys.strip().split()[0].strip()
interval = int(interval.strip())
wf.first_docid = None
wf.thread_event = threading.Event()
wf.thread = socketio.start_background_task(start_procs, wf.userid, keys, interval)
return redirect(url_for('.search_results', keys=keys, interval=interval))
[Less]
|
Posted
over 5 years
ago
by
Jaron Strypsteen
We're using gyroscope on a phone (web) to send the data to a game (web). After some movement and playing all the game stops receiving data from Socket.IO and the backend (FLASK) get the log:
socket.timeout: timed out
[ERROR]
... [More]
Socket error processing request.
I've changed the timeout in gunicorn to 600, I'm using eventlet.
The phone's send data everytime it moves so a lot of requests but it should be able to handle it.
SocketIO:
socketio = SocketIO(app, cors_allowed_origins="*", engineio_logger=False,log_output=False, async_mode='eventlet', debug=False)
Gunicorn command
gunicorn -w 1 --timeout 600 --bind=0.0.0.0 app:app --worker-class eventlet -k eventlet --reload
[Less]
|
Posted
over 5 years
ago
by
Peter S
I run my app test.py on Win10 in the virtual environment venv
I have installled eventlet
My code is:
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
app = Flask(__name__)
app.config['SECRET_KEY'] =
... [More]
'include_help!'
socketio = SocketIO(app)
@app.route('/')
def output():
return render_template('part.html')
@socketio.on('connect')
def app_connect():
print('Client connected');
emit('my_event', {'number': "12345678"})
if __name__ == '__main__':
socketio.run(app)
when I run: "flask run" I get the following result
(vfern) C:\Users\ps\Python\fernb>flask run
* Serving Flask app "test.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
[2020-01-15 18:40:30,941] WARNING in __init__: Flask-SocketIO is Running under Werkzeug, WebSocket is not available.
if i run "python test.py" I get:
File "C:\Users\ps\Python\fernb\socket.py", line 1, in
from flask import Flask, render_template
ImportError: cannot import name 'Flask' from partially initialized module 'flask' (most likely due to a circular import) (C:\Users\ps\Python\fernb\vfern\lib\site-packages\flask\__init__.py)
Any help is appreciated
[Less]
|
Posted
over 5 years
ago
by
Niketan Gulekar
I am developing a Flask application that uses WebSockets ( Flask-SocketsIO ) and Google Firebase. Theres a function which retrieves the user data from firebase, and sends notification to them. If I use flask run to start the web server
... [More]
, everything works fine, including Sockets and the notification method. But, when I use gunicorn -w 1 "app:create_app()" for starting the web server, as soon as the notification sending method is called, the server kinda freezes, and in terminal, it shows following :
[CRITICAL] WORKER TIMEOUT
exception calling callback for
The full error stack is shared here
Note that I can't use multiple workers since I am using Flask-SocketsIO. It doesn't supports multiple workers. Thanks!
[Less]
|
Posted
over 5 years
ago
by
Niketan Gulekar
I am developing a Flask application that uses WebSockets ( Flask-SocketsIO ) and Google Firebase. Theres a function which retrieves the user data from firebase, and sends notification to them. If I use flask run to start the web server
... [More]
, everything works fine, including Sockets and the notification method. But, when I use gunicorn --worker-class eventlet -w 1 "app:create_app()"" for starting the web server, as soon as the notification sending method is called, the server kinda freezes, and in terminal, it shows following :
[CRITICAL] WORKER TIMEOUT
exception calling callback for
The full error stack is shared here
Note that I can't use multiple workers since I am using Flask-SocketsIO. It doesn't supports multiple workers. Thanks!
[Less]
|
Posted
over 5 years
ago
by
Patrick Brodie Wayland
I have looked at a lot of similar problems on message boards and read the related documentation and from what I can tell it seems like I am doing everything right.
node server code
io.on('connection', function(socket){
... [More]
logger.log('info', `\n${Date().toLocaleString('short')}\n:: socket.io client connection established ::\nsocket.id: ${socket.id}\nsocket.handshake.address: ${socket.handshake.address}`)
socket.on('pythonTest', function(){
logger.log('info', `HI THIS IS PYTHON TALKING`)
});
socket.on('userConnect', function(userAgent){
logger.log('info', `\nhttp user connected\n${userAgent}`)
});
});
python client code
import socketio
from pprint import pprint
socket = socketio.Client()
socket.connect('http://localhost:3000')
pprint(vars(socket))
socket.emit('pythonTest', 'pythonTest')
logfile and terminal outputs
When I launch the python script I can see that the socket connection works, however something is wrong with the event 'pythonTest'. The event 'userConnect' behaves as it should, but the emitter for 'userConnect' comes from a web browser which leads me to believe the problem is on the python side of things and not the event handler in node. Im thinking there is specifically something wrong with socket.emit('pythonTest', 'pythonTest') but I have no idea what it could be.
[Less]
|
Posted
over 5 years
ago
by
Patrick Brodie Wayland
I have looked at a lot of similar problems on message boards and read the related documentation and from what I can tell it seems like I am doing everything right.
node server code
io.on('connection', function(socket){
... [More]
logger.log('info', `\n${Date().toLocaleString('short')}\n:: socket.io client connection established ::\nsocket.id: ${socket.id}\nsocket.handshake.address: ${socket.handshake.address}`)
socket.on('pythonTest', function(){
logger.log('info', `HI THIS IS PYTHON TALKING`)
});
socket.on('userConnect', function(userAgent){
logger.log('info', `\nhttp user connected\n${userAgent}`)
});
});
python client code
import socketio
from pprint import pprint
socket = socketio.Client()
socket.connect('http://localhost:3000')
pprint(vars(socket))
socket.emit('pythonTest', 'pythonTest')
logfile and terminal outputs
When I launch the python script I can see that the socket connection works, however something is wrong with the event 'pythonTest'. The event 'userConnect' behaves as it should, but the emitter for 'userConnect' comes from a web browser which leads me to believe the problem is on the python side of things and not the event handler in node. Im thinking there is specifically something wrong with socket.emit('pythonTest', 'pythonTest') but I have no idea what it could be.
[Less]
|
Posted
over 5 years
ago
by
David Mnatsakanyan
Lets say I have two gevent pool groups
gevent_pool_1 = Pool(2)
gevent_pool_2 = Pool(2)
I want to
gevent.sleep(10)
in
gevent_pool_1
which will not block the
gevent_pool_2
|
Posted
over 5 years
ago
by
David Mnatsakanyan
Lets say I have two gevent pool groups
gevent_pool_1 = Pool(2)
gevent_pool_2 = Pool(2)
I want to
gevent.sleep(10)
in
gevent_pool_1
which will not block the
gevent_pool_2
|
Posted
over 5 years
ago
by
Srinithi
With celery, I have created listeners to Redis for getting all write events to Redis. Based on the events, I will trigger celery tasks to migrate data from Redis to DB.
I'm using the eventlet pool along with concurrency of 1000. Also
... [More]
, I'm having 5 celery queues for processing my data.
celery -A proj worker -l info -P eventlet -c 1000 -Q event_queue,vap_queue,client_queue,group_queue,ap_queue
Here, I'm facing the problem like, the listener is able to receive all the write events from Redis and workers are able to receive tasks from the listener. But, celery workers are delaying while processing huge number of data. (For example, I will be receiving 800 tasks per 10 seconds for each queue)
I have tried by increasing concurrency to higher values, changing the pool from eventlet to gevent and prefetch multiplier to 1. Still, My workers are delaying to complete a task.
Can anyone help to solve this? I'm new to celery actually :)
[Less]
|